evse-hub 0.2.3__tar.gz

evse_hub-0.2.3/.github/copilot-instructions.md

@@ -0,0 +1,53 @@
+# Copilot instructions for this repository
+
+Purpose
+- Provide concise, repository-specific guidance so an AI coding agent can be productive immediately.
+
+Big picture
+- This project is a small Python 3.11 daemon that polls Modbus/TCP devices and writes metrics to InfluxDB. See `src/poller/main.py` for the polling orchestration.
+- Polling flow: `main.py` creates periodic tasks that call `poll_modbus_async` (in `src/poller/modbus_device.py`) -> `read_*` functions in `src/poller/modbus_reads.py` -> `write_influx_master` in `src/poller/influx_writes.py`.
+
+Key files and responsibilities
+- `src/poller/main.py`: application entrypoint and task orchestration (`periodic_master`).
+- `src/poller/modbus_device.py`: `ModbusDevice` state, connection management, and exponential backoff logic.
+- `src/poller/modbus_reads.py`: low-level Modbus register reads and helpers (e.g. `_f32_be`, `_read_input_registers`). Use these read functions as blocking callables passed to `poll_modbus_async`.
+- `src/poller/influx_writes.py`: writes to two InfluxDB instances using keys from `secrets.yaml`.
+- `src/poller/async_tools.py`: `run_blocking` helper; uses a shared `ThreadPoolExecutor` (`_EXECUTOR`). Prefer this helper for running blocking calls from async code.
+- `src/poller/config.py`: `load_secrets(path)` reads YAML secrets; keys required by `influx_writes.py` are: `ha_ip`, `ha_influx_username`, `ha_influx_pwd`, `mons_ip`, `mons_influx_username`, `mons_influx_pwd`.
+- `src/poller/wifi.py`: checks association via the `iw` tool; used to skip polls when Wi‑Fi is disconnected.
+
+Conventions & patterns to follow
+- Keep polling logic separate from transport-specific device state: add new device protocols by creating new `Device` classes and `poll_*` functions (see `ModbusDevice` as the pattern).
+- Blocking I/O must be run through `run_blocking(...)` to avoid blocking the event loop. Example: `await run_blocking(write_influx_master, secrets, data, dryrun=True)` (see `main.py`).
+- Read functions used by `poll_modbus_async` are synchronous and should accept `(client, unit) -> dict` and raise on error (see `read_master_loadguard`).
+- Follow the backoff semantics in `modbus_device.py` if adding new polling helpers (respect `dev.next_ok_ts` and `dev.fail_count`).
+
+Running & testing
+- Python version: >=3.11 (declared in `pyproject.toml`).
+- Recommended run (from repository root):
+
+```bash
+PYTHONPATH=src python -m poller.main
+```
+
+- Run tests (from repository root):
+
+```bash
+PYTHONPATH=src pytest -q
+```
+
+External dependencies & environment
+- Code imports not listed in `pyproject.toml`: `pymodbus`, `influxdb` (the InfluxDBClient package), `pyyaml` is declared. Install with pip in a virtualenv as needed.
+- `src/poller/wifi.py` requires the `iw` binary on the host.
+- Secrets file: `secrets.yaml` (loaded by `src/poller/config.py`). Do not commit real credentials — the repo contains a placeholder `secrets.yaml` under `src/`.
+
+Examples for common code changes
+- Add a new Modbus read: implement a function `read_<device>(client, unit) -> dict` in `src/poller/modbus_reads.py` mirroring `read_master_loadguard`, then call it from `main.py` via `poll_modbus_async`.
+- Add an output sink: create a new writer function (blocking) in `src/poller/influx_writes.py` or a new module; call it via `run_blocking` from async code. Respect `dryrun=True` for safe local testing.
+
+Notes for AI agents
+- Prefer minimal, focused diffs. Avoid wide refactors unless the change is necessary and accompanied by tests.
+- When changing I/O or dependency usage, update `pyproject.toml` and document required system packages (e.g., `iw`) in the PR description.
+- Use existing functions as canonical examples: `poll_modbus_async`, `read_master_loadguard`, `write_influx_master`, and `run_blocking`.
+
+If anything in this guidance is unclear or you need more examples, ask for clarification and I will iterate.

evse_hub-0.2.3/.gitignore

@@ -0,0 +1,4 @@
+__pycache__
+*.egg-info*
+tests/*yml
+tests/*json

evse_hub-0.2.3/.gitlab-ci.yml

@@ -0,0 +1,40 @@
+stages:
+  - build
+  - deploy
+
+variables:
+  GIT_DEPTH: 0
+  TWINE_USERNAME: "__token__"
+  TWINE_PASSWORD: $PYPI_TOKEN 
+
+build_package:
+  stage: build
+  before_script:
+    - apt-get update
+    - apt-get install -y --no-install-recommends git
+  script:
+    - python -m venv venv
+    - source venv/bin/activate
+    - pip install --upgrade pip build setuptools setuptools-scm wheel
+    - python -m build --no-isolation
+  artifacts:
+    paths:
+      - dist
+    expire_in: 1 day
+
+
+deploy_to_pypi:
+  stage: deploy
+  only:
+    - tags
+  before_script:
+    - apt-get update
+    - apt-get install -y --no-install-recommends git
+    - python -m venv venv
+    - source venv/bin/activate
+    - pip install --upgrade pip twine build setuptools setuptools-scm wheel
+    - echo $PYPI_API_TOKEN
+  script:
+    - rm -rf dist/
+    - python -m build --no-isolation
+    - twine upload dist/*

evse_hub-0.2.3/PKG-INFO

@@ -0,0 +1,18 @@
+Metadata-Version: 2.4
+Name: evse-hub
+Version: 0.2.3
+Summary: Solar monitoring and control daemons
+Requires-Python: >=3.11
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Requires-Dist: PyYAML>=6
+Requires-Dist: requests>=2
+Requires-Dist: pymodbus>=3.11
+Requires-Dist: influxdb>=5.3
+Requires-Dist: msgpack>=1.0
+Requires-Dist: python-dateutil>=2.9
+Requires-Dist: pydantic>=2.0
+Dynamic: license-file
+
+# README
+

evse_hub-0.2.3/README.md

@@ -0,0 +1,2 @@
+# README
+

evse_hub-0.2.3/TODO.md

@@ -0,0 +1,31 @@
+# Various ToDos
+
+## Watchdog
+
+### Algodue watchdog
+
+```bash
+curl -sS -c /tmp/algodue.cookie \
+  -X POST http://ALGODUE_IP/index.htm \
+  -H 'Content-Type: application/x-www-form-urlencoded' \
+  --data 'user=admin&password=admin'
+```
+
+This stores the auth cookie. Then 
+```bash
+curl -sS -b /tmp/algodue.cookie \
+  -X POST http://ALGODUE_IP/parameters_change.htm \
+  -H 'Content-Type: application/x-www-form-urlencoded' \
+  --data 'addressingType=Static' \
+  --data 'host_name=ETHBOARD' \
+  --data 'IP=192.168.1.249' \
+  --data 'gateway=192.168.1.1' \
+  --data 'mask=255.255.255.0' \
+  --data 'primary_dns=8.8.8.8' \
+  --data 'secondary_dns=8.8.4.4' \
+  --data 'logical_addr=01' \
+  --data 'chbSyncWithNTP=on' \
+  --data 'ntp_server=europe.pool.ntp.org' \
+  --data 'utc_correction=%2B01'
+```
+

evse_hub-0.2.3/docs/architecture.md

@@ -0,0 +1,42 @@
+#  Architecture
+
+## General structure
+
+
+> **Mission statement**
+> 
+> We log building electricity consumption and PV production and allow users to **charge their EV conditioned on PV over-production**. Data for informative user dashboards is written to an influxDB.
+
+### Logging
+
+We need to log from the algoDue load Guard to get the total current (positive or negative) to the house. We also want to get data out of the PV inverter to have the actual production data. We want to know the SOC of the building battery. We poll all EVSEs to know their current state (charging / cable connected / etc).
+
+We also want to know when additional EVSEs come online. Need to think about that later. 
+
+### Control
+
+We want to set a max current on all EVSE:
+
+- lift current limit for immediated charging
+- time-dependent current limit for night-charging
+- set adaptive current depending on solar state (SOC and/or actual over-prxoduction)
+
+
+We also want a watchdog to restart the algoDue in case it stops communicating with the master EVSE (timing issue).
+
+### Software structure
+
+Decision between the following general approaches
+
+- One service fetches all necessary data stores it in a state variable and bases control decisions on those. It also stores the data and a heartbeat to the influxdb and controls the current limits on the EVSEs.
+
+- Serveral services
+    - One only fetches data from buidling (algoDue, inverter, SOC,...). And publishes those via MQTT and/or influxdb. Maybe I sperated this into two more services yet
+        - building stuff
+        - EVSE stuf
+    - One only controls the currents on the EVSEs
+
+I like the one service approach better because it requires less inter-process communication. But I want to get almost all the live data out to my influxdb instances anyway. So, maybe that argument does not hold. I like the multi service variant better because each service is less complex and there is a clear separation between data gathering and storing and control. One thing I do not understand yet: my EVSEs allow communication via modbus registers. Reading via port 502, writing via port 503. Can I separate this into two services, or exactly not? Another thing to think about: the master EVSE provides via it's port 502 registers both access to the EVSE details like cable connected, current charging power, etc., and also the data from the loadguard. Maybe the service which polls the EVSE has a device class which knows the attribute *master-EVSE*.
+
+
+

evse_hub-0.2.3/pyproject.toml

@@ -0,0 +1,39 @@
+
+[project]
+name = "evse-hub"
+dynamic = ["version"]
+description = "Solar monitoring and control daemons"
+readme = "README.md"
+requires-python = ">=3.11"
+
+dependencies = [
+  "PyYAML>=6",
+  "requests>=2",
+  "pymodbus>=3.11",
+  "influxdb>=5.3",
+  "msgpack>=1.0",
+  "python-dateutil>=2.9",
+  "pydantic>=2.0",]
+
+[project.scripts]
+evse-online = "evse_hub.cli.evse_online:main"
+evse-scan = "evse_hub.cli.evse_scan:main"
+
+
+[build-system]
+requires = ["setuptools>=68", "wheel", "setuptools_scm[toml]>=6.0"]
+build-backend = "setuptools.build_meta"
+
+[tool.setuptools]
+package-dir = {"" = "src"}
+
+[tool.setuptools.packages.find]
+where = ["src"]
+
+[tool.pytest.ini_options]
+testpaths = ["tests"]
+
+[tool.setuptools_scm]
+version_scheme = "post-release"
+local_scheme = "no-local-version"
+# fallback_version = "0.0.0"

evse_hub-0.2.3/setup.cfg

@@ -0,0 +1,4 @@
+[egg_info]
+tag_build = 
+tag_date = 0
+

evse_hub-0.2.3/src/evse_hub/cli/evse_online.py

@@ -0,0 +1,270 @@
+#!/usr/bin/env python3
+from __future__ import annotations
+
+import asyncio
+import json
+import logging
+import os
+import random
+import signal
+import time
+from dataclasses import dataclass
+from pathlib import Path
+from typing import Any, Dict, List, Optional, Tuple
+
+import yaml
+
+
+# Locations/configurable via environment (useful for systemd unit files)
+EVSE_YML = Path(os.getenv("EVSE_YML", "evse.yml"))
+STATUS_JSON = Path(os.getenv("STATUS_JSON", "status.json"))
+
+# Configure logging from environment (default INFO). Example: LOG_LEVEL=DEBUG
+LOG_LEVEL_NAME = os.getenv("LOG_LEVEL", "INFO").upper()
+logging.basicConfig(level=LOG_LEVEL_NAME)
+logger = logging.getLogger(__name__)
+
+# If set to a level name (e.g. INFO, DEBUG), and the logger's level is
+# at least that verbose, the status.json payload will also be written to
+# the logger at INFO level. Leave unset to disable.
+_status_level_name = os.getenv("WRITE_STATUS_TO_INFO_LEVEL", "")
+WRITE_STATUS_TO_INFO_LEVEL: int | None = None
+if _status_level_name:
+    try:
+        WRITE_STATUS_TO_INFO_LEVEL = logging._nameToLevel.get(_status_level_name.upper())
+    except Exception:
+        WRITE_STATUS_TO_INFO_LEVEL = None
+
+PORT = 502
+
+# Connect behavior
+CONNECT_TIMEOUT_S = 0.45
+RETRY_ONCE_DELAY_S_RANGE = (0.08, 0.25)  # jitter before the retry attempt
+
+# Scheduler behavior
+# How often to re-check *online* devices. Configurable via env var
+# `EVSE_POLL_INTERVAL_S` (seconds). Default: 3600 (1 hour).
+try:
+    BASE_OK_INTERVAL_S = float(os.getenv("EVSE_POLL_INTERVAL_S", "3600"))
+except Exception:
+    BASE_OK_INTERVAL_S = 3600.0
+if BASE_OK_INTERVAL_S <= 0:
+    raise ValueError("EVSE_POLL_INTERVAL_S must be a positive number of seconds")
+
+BASE_FAIL_INTERVAL_S = 60.0              # initial re-check interval after first failure
+MAX_FAIL_INTERVAL_S = 300.0             # cap offline backoff at 5 minutes
+JITTER_FRACTION = 0.15                  # +/- 15% jitter on scheduling
+
+# Concurrency limit (even if you have many devices)
+CONCURRENCY = 50
+
+
+@dataclass
+class DeviceState:
+    unit_id: int
+    name: str
+    ip: Optional[str] = None
+
+    online: bool = False
+    fails: int = 0                    # consecutive failures
+    next_check_ts: float = 0.0        # monotonic time when to check next
+
+
+def _with_jitter(seconds: float, frac: float = JITTER_FRACTION) -> float:
+    """
+    Apply multiplicative jitter: seconds * U(1-frac, 1+frac)
+    """
+    lo = max(0.0, 1.0 - frac)
+    hi = 1.0 + frac
+    return seconds * random.uniform(lo, hi)
+
+
+async def tcp_connect_ok(ip: str, port: int, timeout_s: float) -> bool:
+    """
+    True if TCP connect succeeds within timeout; closes immediately.
+    """
+    try:
+        reader, writer = await asyncio.wait_for(asyncio.open_connection(ip, port), timeout=timeout_s)
+        writer.close()
+        try:
+            await writer.wait_closed()
+        except Exception:
+            pass
+        return True
+    except (asyncio.TimeoutError, OSError):
+        return False
+
+
+async def check_with_retry(ip: str) -> bool:
+    """
+    Try once; if it fails, wait a small jittered delay and try once more.
+    """
+    ok = await tcp_connect_ok(ip, PORT, CONNECT_TIMEOUT_S)
+    if ok:
+        return True
+
+    await asyncio.sleep(random.uniform(*RETRY_ONCE_DELAY_S_RANGE))
+    return await tcp_connect_ok(ip, PORT, CONNECT_TIMEOUT_S)
+
+
+def load_devices(path: Path) -> List[Tuple[int, str, Optional[str]]]:
+    """
+    Returns list of (unit_id, name, ip) from evse.yml devices.
+    `ip` may be `None` when not present in the YAML; such devices will be
+    included in the returned list but not actively checked.
+    """
+    data = yaml.safe_load(path.read_text(encoding="utf-8"))
+    if not isinstance(data, dict):
+        raise ValueError("evse.yml root must be a mapping/dict")
+
+    devices = data.get("devices", [])
+    if not isinstance(devices, list):
+        raise ValueError("evse.yml: 'devices' must be a list")
+
+    out: List[Tuple[int, str, Optional[str]]] = []
+    for d in devices:
+        if not isinstance(d, dict):
+            continue
+        unit_id = d.get("unit_id")
+        name = d.get("name")
+        ip = d.get("ip")
+        if isinstance(unit_id, int) and isinstance(name, str):
+            out.append((unit_id, name, ip if isinstance(ip, str) else None))
+    return out
+
+
+def compute_next_interval(online: bool, fails: int) -> float:
+    """
+    Online: check every BASE_OK_INTERVAL_S.
+    Offline: exponential backoff starting at BASE_FAIL_INTERVAL_S, doubling per consecutive failure,
+             capped at MAX_FAIL_INTERVAL_S.
+    """
+    if online:
+        return BASE_OK_INTERVAL_S
+    # fails is >= 1 here
+    interval = BASE_FAIL_INTERVAL_S * (2 ** max(0, fails - 1))
+    interval = min(interval, MAX_FAIL_INTERVAL_S)
+    return interval
+
+
+def write_status_json(path: Path, states: Dict[int, DeviceState]) -> Dict[str, Any]:
+    """
+    Writes:
+      {
+        "ts": <unix>,
+        "<unit_id>": {"online": true/false},
+        ...
+      }
+    """
+    payload: Dict[str, Any] = {"ts": int(time.time())}
+    for unit_id in sorted(states.keys()):
+        payload[str(unit_id)] = {"online": bool(states[unit_id].online)}
+    path.write_text(json.dumps(payload, indent=2, sort_keys=False) + "\n", encoding="utf-8")
+    return payload
+
+
+async def poll_loop(stop_event: asyncio.Event) -> None:
+    # Load config once at start (easy to add reload-on-change later)
+    devs = load_devices(EVSE_YML)
+    if not devs:
+        raise SystemExit("No devices found under 'devices:' in evse.yml")
+
+    # Create state per unit_id
+    states: Dict[int, DeviceState] = {}
+    now_mono = asyncio.get_running_loop().time()
+    for unit_id, name, ip in devs:
+        states[unit_id] = DeviceState(
+            unit_id=unit_id,
+            name=name,
+            ip=ip,
+            online=False,
+            fails=0,
+            next_check_ts=now_mono + random.uniform(0.0, 1.0) if ip is not None else float("inf"),  # small initial jitter
+        )
+
+    sem = asyncio.Semaphore(CONCURRENCY)
+
+    async def check_one(st: DeviceState) -> None:
+        # If the device has no IP configured, never attempt to check it.
+        if st.ip is None:
+            st.online = False
+            st.fails = 0
+            st.next_check_ts = float("inf")
+            return
+        async with sem:
+            ok = await check_with_retry(st.ip)
+
+        if ok:
+            st.online = True
+            st.fails = 0
+        else:
+            st.online = False
+            st.fails += 1
+
+        interval = compute_next_interval(st.online, max(1, st.fails) if not st.online else 0)
+        interval = _with_jitter(interval)
+        st.next_check_ts = asyncio.get_running_loop().time() + interval
+
+    # Main scheduler loop
+    while not stop_event.is_set():
+        loop = asyncio.get_running_loop()
+        now = loop.time()
+
+        due = [st for st in states.values() if st.next_check_ts <= now]
+        if due:
+            # Kick all due checks concurrently
+            await asyncio.gather(*(check_one(st) for st in due))
+
+            # Update status.json after each batch of checks
+            payload = write_status_json(STATUS_JSON, states)
+
+            # Optionally also write the payload to the logger at INFO
+            # when the configured level is enabled (useful for CI/debug).
+            if WRITE_STATUS_TO_INFO_LEVEL is not None and logger.isEnabledFor(WRITE_STATUS_TO_INFO_LEVEL):
+                logger.info("status.json: %s", json.dumps(payload, sort_keys=True))
+
+        # Sleep until the next device is due (or a short minimum to avoid busy loop)
+        next_due = min(st.next_check_ts for st in states.values())
+        sleep_s = max(0.1, next_due - loop.time())
+
+        # Wait either for the sleep to finish or for a shutdown signal
+        sleep_task = asyncio.create_task(asyncio.sleep(sleep_s))
+        stop_task = asyncio.create_task(stop_event.wait())
+        done, pending = await asyncio.wait({sleep_task, stop_task}, return_when=asyncio.FIRST_COMPLETED)
+        for p in pending:
+            p.cancel()
+
+    # On shutdown request, write one final status snapshot
+    try:
+        payload = write_status_json(STATUS_JSON, states)
+        if WRITE_STATUS_TO_INFO_LEVEL is not None and logger.isEnabledFor(WRITE_STATUS_TO_INFO_LEVEL):
+            logger.info("final status.json: %s", json.dumps(payload, sort_keys=True))
+    except Exception:
+        logger.exception("Failed writing final status.json")
+
+
+def main() -> None:
+    # Create an asyncio event and attach signal handlers for graceful shutdown
+    async def _main_async() -> None:
+        loop = asyncio.get_running_loop()
+        stop_event = asyncio.Event()
+
+        # Register handlers to set the stop event on SIGINT/SIGTERM
+        try:
+            loop.add_signal_handler(signal.SIGINT, stop_event.set)
+            loop.add_signal_handler(signal.SIGTERM, stop_event.set)
+        except NotImplementedError:
+            # Some platforms (or Python runtimes) may not support add_signal_handler
+            pass
+
+        await poll_loop(stop_event)
+
+    try:
+        asyncio.run(_main_async())
+    except KeyboardInterrupt:
+        # asyncio signal handlers should already set the stop event; ensure exit
+        logger.info("Interrupted by user; exiting")
+
+
+if __name__ == "__main__":
+    main()

evse_hub-0.2.3/src/evse_hub/cli/evse_scan.py

@@ -0,0 +1,75 @@
+#!/usr/bin/env python3
+from __future__ import annotations
+
+import asyncio
+import ipaddress
+from dataclasses import dataclass
+from typing import List
+
+
+@dataclass(frozen=True)
+class HostResult:
+    ip: str
+    port_502_open: bool
+
+
+async def check_tcp_port(ip: str, port: int, timeout_s: float) -> bool:
+    """
+    True if TCP connect succeeds within timeout.
+    """
+    try:
+        conn = asyncio.open_connection(ip, port)
+        reader, writer = await asyncio.wait_for(conn, timeout=timeout_s)
+        writer.close()
+        # Python 3.11+: wait_closed exists for StreamWriter
+        try:
+            await writer.wait_closed()
+        except Exception:
+            pass
+        return True
+    except (asyncio.TimeoutError, OSError):
+        return False
+
+
+async def scan_modbus_502_async(
+    cidr: str = "192.168.1.0/24",
+    timeout_s: float = 0.35,
+    concurrency: int = 400,
+) -> List[HostResult]:
+    net = ipaddress.ip_network(cidr, strict=False)
+    ips = [str(ip) for ip in net.hosts()]
+
+    sem = asyncio.Semaphore(concurrency)
+
+    async def one(ip: str) -> HostResult:
+        async with sem:
+            ok = await check_tcp_port(ip, 502, timeout_s)
+            return HostResult(ip=ip, port_502_open=ok)
+
+    results = await asyncio.gather(*(one(ip) for ip in ips))
+    hits = [r for r in results if r.port_502_open]
+    hits.sort(key=lambda r: ipaddress.ip_address(r.ip))
+    return hits
+
+
+def main() -> None:
+    async def _main_async() -> None:
+        
+        hits = await scan_modbus_502_async()
+        if not hits:
+            print("No hosts with TCP/502 open found in 192.168.1.0/24")
+            return
+
+        print("Hosts with TCP/502 open:")
+        for h in hits:
+            print(f"  {h.ip}:502")
+
+    try:
+        asyncio.run(_main_async())
+    except KeyboardInterrupt:
+        # asyncio signal handlers should already set the stop event; ensure exit
+        print("Interrupted by user; exiting")
+
+
+if __name__ == "__main__":
+    main()

evse_hub-0.2.3/src/evse_hub/poller/TODO.md

@@ -0,0 +1,37 @@
+# TODO
+
+- [x] [influx_writes.py](influx_writes.py) actually need the wifi checking
+
+## What do I need to understand
+
+- [x] [aync_tools.py](async_tools.py)
+    - Creates a pool of threads for asyncio
+    - Defines pool size
+    - relays `*args` and `**kwargs` to passed function 
+- [x] [config.py](config.py)
+    - Reads in configurations from a file `secrets.yml`
+- [x] [device.py](device.py)
+    - I understood basic principles of hirachical loggin
+    - I understood that `@dataclass` just produces standard `__init__` and other methods for classes that are mostly data (think of it as dict+)
+    - `poll_modbus_async` can check for wifi and has exponential back-off implemented. Hopefully not needed now that `ceorl` is in ethernet of garage. 
+- [x] [influx_writes.py](influx_writes.py)
+    - writes messages to influxdb on ha and mons
+- [x] [main.py](main.py)
+    - defines logging level
+    - defines the functions which will be run in the loop
+    - inside the main
+        - Creates a list of devices
+        - Creates a list of tasks
+        - Runs the tasks in a loop
+- [x] [modbus_reads.py](modbus_reads.py)
+    - knows how to cast binary stuff into readable messages
+    - Has functions that poll specific things. 
+        - Building power / currents
+        - ..
+
+        Each of those returns a dict that [influx_writes.py](influx_writes.py) will know what to do.
+- [x] [wifi.py](wifi.py)
+    - checks via `iw` if wifi link is healthy ( `iw` must be present in system!)
+
+---
+Overall: main.py imports devices, devices have the connection as member, for a device there needs to be function which unpacks that connection, "*_reads.py" and "*_writes.py" will use that connection. 

evse_hub-0.2.3/src/evse_hub/poller/async_tools.py

@@ -0,0 +1,27 @@
+from __future__ import annotations
+
+import asyncio
+from concurrent.futures import ThreadPoolExecutor
+from typing import Any, Callable
+
+# Small, shared pool for all blocking work (modbus + influx + subprocess).
+# Tune max_workers if you add many devices; 4–8 is typically plenty here.
+_EXECUTOR = ThreadPoolExecutor(max_workers=4)
+
+
+async def run_blocking(fn: Callable[..., Any], /, *args: Any, timeout_s: float | None = None, **kwargs: Any) -> Any:
+    """
+    Run a blocking function in a bounded thread pool, optionally with a timeout.
+
+    This is a thin wrapper around loop.run_in_executor (aka to_thread with control).
+    """
+    loop = asyncio.get_running_loop()
+
+    def _call():
+        return fn(*args, **kwargs)
+
+    if timeout_s is None:
+        return await loop.run_in_executor(_EXECUTOR, _call)
+
+    async with asyncio.timeout(timeout_s):
+        return await loop.run_in_executor(_EXECUTOR, _call)

evse_hub-0.2.3/src/evse_hub/poller/config.py

@@ -0,0 +1,7 @@
+from __future__ import annotations
+import yaml
+
+
+def load_secrets(path: str = "secrets.yaml") -> dict:
+    with open(path, "r") as f:
+        return yaml.safe_load(f)

evse_hub-0.2.3/src/evse_hub/poller/etrel.py

@@ -0,0 +1,41 @@
+"""Simple lookup table for Etrel connector status codes."""
+
+CONNECTOR_STATUS: dict[int, str] = {
+    0: "Unknown",
+    1: "SocketAvailable",
+    2: "WaitingForVehicleToBeConnected",
+    3: "WaitingForVehicleToStart",
+    4: "Charging",
+    5: "ChargingPausedByEv",
+    6: "ChargingPausedByEvse",
+    7: "ChargingEnded",
+    8: "ChargingFault",
+    9: "UnpausingCharging",
+    10: "Unavailable",
+}
+
+def get_connector_status(code: int) -> str:
+    """Return the human-readable status for a connector code."""
+    return CONNECTOR_STATUS.get(code, f"Unknown({code})")
+
+CONNECTOR_TYPE: dict[int, str] = {
+    1: "SocketType",
+    2: "PlugType",
+}
+
+def get_connector_type(code: int) -> str:
+    """Return the human-readable connector type for a code."""
+    return CONNECTOR_TYPE.get(code, f"Unknown({code})")
+
+MEASURED_PHASES: dict[int, str] = {
+    0: "Three phases",
+    1: "Single phase L1",
+    2: "Single phase L2",
+    3: "Single phase L3",
+    4: "Unknown",
+    5: "Two phases",
+}
+
+def get_measured_phases(code: int) -> str:
+    """Return the human-readable measured phases for a code."""
+    return MEASURED_PHASES.get(code, f"Unknown({code})")

evse_hub-0.2.3/src/evse_hub/poller/influx_writes.py

@@ -0,0 +1,103 @@
+from __future__ import annotations
+
+from influxdb import InfluxDBClient
+import logging
+log = logging.getLogger(__name__)
+
+
+def write_influx_master(secrets: dict, data: dict, dryrun: bool = False) -> dict:
+    """
+    Write measurements to two InfluxDB instances.
+
+    Returns a dict with per-host success booleans: {"ha": bool, "mons": bool}.
+    """
+    statuses = {"ha": False, "mons": False}
+
+    # 1) HA influx (building_power DB)
+    try:
+        payload_ha = {
+            "measurement": "power",
+            "fields": {"kW": data["p_total_kW"]},
+            "tags": {"success": "1"},
+        }
+
+        if dryrun:
+            log.info("HA dryrun payload: %s", payload_ha)
+            statuses["ha"] = True
+        else:
+            try:
+                ha = InfluxDBClient(
+                    host=secrets["ha_ip"], port=8086,
+                    username=secrets["ha_influx_username"],
+                    password=secrets["ha_influx_pwd"],
+                    timeout=5,
+                )
+            except Exception:
+                log.exception("Creating HA InfluxDB client failed")
+                ha = None
+
+            if ha is not None:
+                try:
+                    ha.switch_database("building_power")
+                    ha.write_points([payload_ha])
+                    statuses["ha"] = True
+                except Exception:
+                    log.exception("Writing to HA influx failed")
+                finally:
+                    try:
+                        ha.close()
+                    except Exception:
+                        pass
+
+    except Exception:
+        log.exception("Unexpected error preparing HA payload")
+
+    # 2) mons influx (building DB)
+    try:
+        payload_mons = {
+            "measurement": "power",
+            "fields": {
+                "power_kW": data["p_total_kW"],
+                "current_L1_A": data["i_l1_A"],
+                "current_L2_A": data["i_l2_A"],
+                "current_L3_A": data["i_l3_A"],
+                "voltage_L1N_V": data["u_l1_V"],
+                "voltage_L2N_V": data["u_l2_V"],
+                "voltage_L3N_V": data["u_l3_V"],
+                "lg_connected": int(data["lg_connected"]) if data["lg_connected"] is not None else -1,
+            },
+            "tags": {"asset": "hak", "flow": "bidirectional"},
+        }
+
+        if dryrun:
+            log.info("mons dryrun payload: %s", payload_mons)
+            statuses["mons"] = True
+        else:
+            try:
+                mons = InfluxDBClient(
+                    host=secrets["mons_ip"], port=8086,
+                    username=secrets["mons_influx_username"],
+                    password=secrets["mons_influx_pwd"],
+                    timeout=5,
+                )
+            except Exception:
+                log.exception("Creating mons InfluxDB client failed")
+                mons = None
+
+            if mons is not None:
+                try:
+                    mons.switch_database("building")
+                    mons.write_points([payload_mons])
+                    statuses["mons"] = True
+                except Exception:
+                    log.exception("Writing to mons influx failed")
+                finally:
+                    try:
+                        mons.close()
+                    except Exception:
+                        pass
+
+    except Exception:
+        log.exception("Unexpected error preparing mons payload")
+
+    return statuses

evse_hub-0.2.3/src/evse_hub/poller/main.py

@@ -0,0 +1,84 @@
+from __future__ import annotations
+
+import asyncio
+import logging
+from typing import Dict
+
+from .config import load_secrets
+from .modbus_device import ModbusDevice, poll_modbus_async
+from .modbus_reads import read_master_loadguard
+from .influx_writes import write_influx_master
+from .async_tools import run_blocking
+
+logging.basicConfig(level=logging.INFO)
+log = logging.getLogger(__name__)
+
+
+async def periodic_master(dev: ModbusDevice, secrets: dict, period_s: float) -> None:
+    try:
+        while True:
+            data = await poll_modbus_async(dev, read_master_loadguard)
+            if data is not None:
+                await run_blocking(
+                    write_influx_master,
+                    secrets,
+                    data,
+                    dryrun=True,
+                    timeout_s=5.0,
+                )
+            await asyncio.sleep(period_s)
+    except asyncio.CancelledError:
+        # Task shutdown is expected (Ctrl-C / cancellation); exit quietly.
+        return
+
+
+async def main() -> None:
+    secrets = load_secrets("secrets.yaml")
+
+    mb_devices: Dict[str, ModbusDevice] = {
+        "master_etrel": ModbusDevice("master_etrel", "192.168.1.121", 503, timeout_s=2.0),
+    }
+
+    tasks = [
+        asyncio.create_task(
+            periodic_master(mb_devices["master_etrel"], secrets, period_s=15.0),
+            name="periodic_master",
+        ),
+    ]
+
+    try:
+        await asyncio.gather(*tasks)
+
+    except asyncio.CancelledError:
+        # If the event loop cancels us, proceed to cleanup.
+        pass
+
+    finally:
+        log.info("Shutting down...")
+
+        for t in tasks:
+            t.cancel()
+        await asyncio.gather(*tasks, return_exceptions=True)
+
+        for d in mb_devices.values():
+            d.close()
+
+        log.info("Shutdown complete.")
+
+
+if __name__ == "__main__":
+    try:
+        asyncio.run(main())
+    except KeyboardInterrupt:
+        # Swallow the final KeyboardInterrupt that asyncio.run may raise on SIGINT.
+        pass
+
+
+def cli() -> None:
+    try:
+        asyncio.run(main())
+    except KeyboardInterrupt:
+        pass
+
+if __name__ == "__main__":
+    cli()

evse_hub-0.2.3/src/evse_hub/poller/modbus_device.py

@@ -0,0 +1,92 @@
+from __future__ import annotations
+
+import logging
+import time
+from dataclasses import dataclass
+from typing import Callable, Optional
+
+from pymodbus.client import ModbusTcpClient
+
+from .wifi import wifi_is_associated
+from .async_tools import run_blocking
+
+log = logging.getLogger(__name__)
+
+
+@dataclass
+class ModbusDevice:
+    """
+    A Modbus/TCP endpoint + polling backoff state.
+
+    Keep this intentionally Modbus-specific. If/when you add MQTT or HTTP-polled
+    devices, create separate *Device + poll_* functions rather than overloading
+    this class with protocol-specific fields.
+    """
+    name: str
+    host: str
+    port: int
+    unit: int = 1
+    timeout_s: float = 2.0
+
+    client: Optional[ModbusTcpClient] = None
+    next_ok_ts: float = 0.0
+    fail_count: int = 0
+
+    def ensure_client(self) -> ModbusTcpClient:
+        if self.client is None:
+            self.client = ModbusTcpClient(self.host, port=self.port, timeout=self.timeout_s)
+        return self.client
+
+    def close(self) -> None:
+        if self.client is not None:
+            try:
+                self.client.close()
+            except Exception:
+                pass
+        self.client = None
+
+
+async def poll_modbus_async(
+    dev: ModbusDevice,
+    read_fn: Callable[[ModbusTcpClient, int], dict],
+    *,
+    wifi_if: str = "wlan0",
+) -> Optional[dict]:
+    """
+    Poll a Modbus device with exponential backoff, returning measurements on success.
+
+    `read_fn` is your existing blocking block-read function:
+        read_fn(client: ModbusTcpClient, unit: int) -> dict
+    """
+    now = time.time()
+    if now < dev.next_ok_ts:
+        return None
+
+    if wifi_if:
+        associated = await run_blocking(wifi_is_associated, wifi_if, timeout_s=1.5)
+        if not associated:
+            log.warning("Wi-Fi %s not associated; skipping poll for %s", wifi_if, dev.name)
+            # Try again soon; don't "punish" the device with exponential backoff.
+            dev.next_ok_ts = now + 2.0
+            return None
+
+    client = dev.ensure_client()
+
+    try:
+        ok = await run_blocking(client.connect, timeout_s=dev.timeout_s + 1.0)
+        if not ok:
+            raise RuntimeError("connect failed")
+
+        data = await run_blocking(read_fn, client, dev.unit, timeout_s=dev.timeout_s + 2.0)
+
+        dev.fail_count = 0
+        dev.next_ok_ts = now
+        return data
+
+    except Exception as e:
+        dev.fail_count += 1
+        backoff = min(60, 2 ** min(dev.fail_count, 6))
+        dev.next_ok_ts = now + backoff
+        log.warning("%s poll failed (%s). backoff=%ss", dev.name, e, backoff)
+        dev.close()
+        return None

evse_hub-0.2.3/src/evse_hub/poller/modbus_reads.py

@@ -0,0 +1,95 @@
+from __future__ import annotations
+
+from struct import pack, unpack
+from pymodbus.client import ModbusTcpClient
+
+
+def _f32_be(regs: list[int], idx: int) -> float:
+    raw = pack(">HH", int(regs[idx]) & 0xFFFF, int(regs[idx + 1]) & 0xFFFF)
+    return float(unpack(">f", raw)[0])
+
+
+def _i64_be(regs: list[int], idx: int) -> int:
+    raw = pack(
+        ">HHHH",
+        int(regs[idx]) & 0xFFFF,
+        int(regs[idx + 1]) & 0xFFFF,
+        int(regs[idx + 2]) & 0xFFFF,
+        int(regs[idx + 3]) & 0xFFFF,
+    )
+    return int(unpack(">q", raw)[0])
+
+def _u32_be(regs: list[int], idx: int) -> int:
+    raw = pack(">HH", int(regs[idx + 1]) & 0xFFFF, int(regs[idx]) & 0xFFFF)
+    return int(unpack(">i", raw)[0])
+
+
+def _ascii_be(regs: list[int], idx: int, n_regs: int, *, strip_null: bool = True) -> str:
+    b = bytearray()
+    for r in regs[idx: idx + n_regs]:
+        r &= 0xFFFF
+        b.append((r >> 8) & 0xFF)   # high byte
+        b.append(r & 0xFF)          # low byte
+
+    if strip_null:
+        b = b.split(b"\x00", 1)[0]  # stop at first NUL
+
+    return b.decode("ascii", errors="replace").rstrip()
+
+def _read_input_registers(client: ModbusTcpClient, *, address: int, count: int, unit_id: int):
+    """
+    pymodbus kwarg name varies by version: slave / unit / device_id / none.
+    Keep it simple and just try the common ones.
+    """
+    fn = client.read_input_registers
+
+    # Try keyword variations first
+    for kw in ("slave", "unit", "device_id"):
+        try:
+            return fn(address=address, count=count, **{kw: unit_id})
+        except TypeError:
+            pass
+
+    # Fallback: no unit parameter (some setups default to unit 1)
+    return fn(address=address, count=count)
+
+
+def read_master_loadguard(client: ModbusTcpClient, unit: int) -> dict:
+    rr = _read_input_registers(client, address=2000, count=26, unit_id=unit)
+    if rr.isError():
+        raise RuntimeError(rr)
+
+    r = rr.registers
+
+    lg_raw = int(r[0])
+    if lg_raw == 0:
+        lg_status = "Not connected"
+        lg_connected = False
+    elif lg_raw == 1:
+        lg_status = "Connected"
+        lg_connected = True
+    else:
+        lg_status = "Unknown connection status!"
+        lg_connected = None
+
+    u_l1 = _f32_be(r, 2004 - 2000)
+    u_l2 = _f32_be(r, 2006 - 2000)
+    u_l3 = _f32_be(r, 2008 - 2000)
+
+    i_l1 = _f32_be(r, 2010 - 2000)
+    i_l2 = _f32_be(r, 2012 - 2000)
+    i_l3 = _f32_be(r, 2014 - 2000)
+
+    p_tot = _f32_be(r, 2022 - 2000)
+
+    return {
+        "lg_connected": lg_connected,
+        "lg_status": lg_status,
+        "u_l1_V": round(u_l1, 2),
+        "u_l2_V": round(u_l2, 2),
+        "u_l3_V": round(u_l3, 2),
+        "i_l1_A": round(i_l1, 2),
+        "i_l2_A": round(i_l2, 2),
+        "i_l3_A": round(i_l3, 2),
+        "p_total_kW": round(p_tot, 2),
+    }

evse_hub-0.2.3/src/evse_hub/poller/wifi.py

@@ -0,0 +1,20 @@
+from __future__ import annotations
+
+import subprocess
+
+def wifi_is_associated(ifname: str = "wlan0") -> bool:
+    """
+    True if the kernel reports the interface is associated to an AP.
+    Requires: iw
+    """
+    try:
+        out = subprocess.check_output(
+            ["iw", "dev", ifname, "link"],
+            text=True,
+            stderr=subprocess.STDOUT,
+            timeout=1.0,
+        )
+    except Exception:
+        return False
+
+    return out.startswith("Connected to ")

evse_hub-0.2.3/src/evse_hub.egg-info/PKG-INFO

@@ -0,0 +1,18 @@
+Metadata-Version: 2.4
+Name: evse-hub
+Version: 0.2.3
+Summary: Solar monitoring and control daemons
+Requires-Python: >=3.11
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Requires-Dist: PyYAML>=6
+Requires-Dist: requests>=2
+Requires-Dist: pymodbus>=3.11
+Requires-Dist: influxdb>=5.3
+Requires-Dist: msgpack>=1.0
+Requires-Dist: python-dateutil>=2.9
+Requires-Dist: pydantic>=2.0
+Dynamic: license-file
+
+# README
+

evse_hub-0.2.3/src/evse_hub.egg-info/SOURCES.txt

@@ -0,0 +1,30 @@
+.gitignore
+.gitlab-ci.yml
+LICENSE
+README.md
+TODO.md
+pyproject.toml
+secrets.yaml
+.github/copilot-instructions.md
+docs/architecture.md
+docs/operations.md
+src/evse_hub.egg-info/PKG-INFO
+src/evse_hub.egg-info/SOURCES.txt
+src/evse_hub.egg-info/dependency_links.txt
+src/evse_hub.egg-info/entry_points.txt
+src/evse_hub.egg-info/requires.txt
+src/evse_hub.egg-info/top_level.txt
+src/evse_hub/cli/evse_online.py
+src/evse_hub/cli/evse_scan.py
+src/evse_hub/poller/TODO.md
+src/evse_hub/poller/__init__.py
+src/evse_hub/poller/async_tools.py
+src/evse_hub/poller/config.py
+src/evse_hub/poller/etrel.py
+src/evse_hub/poller/influx_writes.py
+src/evse_hub/poller/main.py
+src/evse_hub/poller/modbus_device.py
+src/evse_hub/poller/modbus_reads.py
+src/evse_hub/poller/wifi.py
+tests/read_registers.py
+tests/test_asyncio.py

evse_hub-0.2.3/src/evse_hub.egg-info/dependency_links.txt

@@ -0,0 +1 @@
+

evse_hub-0.2.3/src/evse_hub.egg-info/entry_points.txt

@@ -0,0 +1,3 @@
+[console_scripts]
+evse-online = evse_hub.cli.evse_online:main
+evse-scan = evse_hub.cli.evse_scan:main

evse_hub-0.2.3/src/evse_hub.egg-info/requires.txt

@@ -0,0 +1,7 @@
+PyYAML>=6
+requests>=2
+pymodbus>=3.11
+influxdb>=5.3
+msgpack>=1.0
+python-dateutil>=2.9
+pydantic>=2.0

evse_hub-0.2.3/src/evse_hub.egg-info/top_level.txt

@@ -0,0 +1 @@
+evse_hub

evse_hub-0.2.3/tests/read_registers.py

@@ -0,0 +1,84 @@
+from evse_hub.poller.modbus_device import ModbusDevice
+from evse_hub.poller.modbus_reads import _read_input_registers, _f32_be, _ascii_be, _i64_be, _u32_be
+from evse_hub.poller.config import load_secrets
+from evse_hub.poller.etrel import *
+import datetime
+
+evses=load_secrets("evse.yaml")['devices']
+
+for evse in evses:
+    try:
+        
+        dev=ModbusDevice(evse["name"], evse["ip"], 502, timeout_s=2.0)
+        client = dev.ensure_client()
+        ok=client.connect()
+        print(f"\n{evse["name"]} connected: {ok}\n")
+        rr=_read_input_registers(client, address=0, count=48, unit_id=1)
+        r=rr.registers
+        print(f"Connector status: {CONNECTOR_STATUS[r[0]]}")
+        print(f"Connector measured number of phases: {MEASURED_PHASES[r[1]]}")
+        print(f"EV max phase current: {_f32_be(r, 2)} A")
+        print(f"Target current from power mgm or modbus: {_f32_be(r, 4)} A")
+        print(f"Frequency: {_f32_be(r, 6):.3f} Hz")
+        print(f"L1-N Voltage: {_f32_be(r, 8):.1f} V")
+
+        print(f"L2-N Voltage: {_f32_be(r, 10):.1f} V")
+        print(f"L3-N Voltage: {_f32_be(r, 12):.1f} V")
+        print(f"L1 Current: {_f32_be(r, 14):.2f} A")
+        print(f"L2 Current: {_f32_be(r, 16):.2f} A")
+        print(f"L3 Current: {_f32_be(r, 18):.2f} A")
+        print(f"Active power L1: {_f32_be(r, 20):.2f} kW")
+        print(f"Active power L2: {_f32_be(r, 22):.2f} kW")
+        print(f"Active power L3: {_f32_be(r, 24):.2f} kW")
+        print(f"Total power: {_f32_be(r, 26):.2f} kW")
+        print(f"Total imported active energy in running session: {_f32_be(r, 30):.2f} kWh")
+        print(f"Running session max power: {_f32_be(r, 44):.2f} kW")
+
+        rr=_read_input_registers(client, address=990, count=40, unit_id=1)
+        r=rr.registers
+        print(f"Serial number: {(_ascii_be(r, 0, 10))}")
+        print(f"Model: {(_ascii_be(r, 1000-990, 10))}")
+        print(f"HW Version: {(_ascii_be(r, 1010-990, 5))}")
+        print(f"SW Version: {(_ascii_be(r, 1015-990, 5))}")
+        print(f"Number of connectors: {_u32_be(r, 1020-990)}")
+        print(f"Connector type: {CONNECTOR_TYPE[r[1022-990]]}")
+        print(f"Number of phases: {r[1023-990]}")
+        print(f"L1 connected to: {r[1024-990]}")
+        print(f"L2 connected to: {r[1025-990]}")
+        print(f"L3 connected to: {r[1026-990]}")
+        print(f"Custom max current: {_f32_be(r, 1028-990)} A")
+        client.close()
+        if evse['type'] == 'master':
+            dev=ModbusDevice(evse["name"], evse["ip"], 503, timeout_s=2.0)
+            client = dev.ensure_client()
+            ok=client.connect()
+            print(f"\n{evse["name"]} connected: {ok}\n")
+            rr=_read_input_registers(client, address=3000, count=1, unit_id=1)
+            r=rr.registers
+            print(f"Loadguard installed: {r[0]}")
+            rr=_read_input_registers(client, address=2000, count=26, unit_id=1)
+            r=rr.registers
+            print(f"LoadGuard connected: {r[0]}")
+            print(f"Frequency: {_f32_be(r, 2002-2000):.2f} Hz")
+            print(f"L1-N Voltage: {_f32_be(r, 2004-2000):.1f} V")
+            print(f"L2-N Voltage: {_f32_be(r, 2006-2000):.1f} V")
+            print(f"L3-N Voltage: {_f32_be(r, 2008-2000):.1f} V")
+            print(f"L1 Current: {_f32_be(r, 2010-2000):.2f} A")
+            print(f"L2 Current: {_f32_be(r, 2012-2000):.2f} A")
+            print(f"L3 Current: {_f32_be(r, 2014-2000):.2f} A")
+            print(f"Active power L 1: {_f32_be(r, 2016-2000):.2f} kW")
+            print(f"Active power L 2: {_f32_be(r, 2018-2000):.2f} kW")
+            print(f"Active power L 3: {_f32_be(r, 2020-2000):.2f} kW")
+            print(f"Total power: {_f32_be(r, 2022-2000):.2f} kW")
+            rr=_read_input_registers(client, address=2100, count=14, unit_id=1)
+            r=rr.registers
+            print(f"Power cluster Current L1: {_f32_be(r, 2100-2100):.2f} A")
+            print(f"Power cluster Current L2: {_f32_be(r, 2102-2100):.2f} A")
+            print(f"Power cluster Current L3: {_f32_be(r, 2104-2100):.2f} A")
+            print(f"Power cluster Active power L1: {_f32_be(r, 2106-2100):.2f} kW")
+            print(f"Power cluster Active power L2: {_f32_be(r, 2108-2100):.2f} kW")
+            print(f"Power cluster Active power L3: {_f32_be(r, 2110-2100):.2f} kW")
+            print(f"Power cluster Total power: {_f32_be(r, 2112-2100):.2f} kW")
+            client.close()
+    except Exception as e:
+        print(f"Error reading {evse['name']}: {e}")