somestuff-mqttwebstuff 0.1.4__tar.gz

somestuff_mqttwebstuff-0.1.4/.gitignore

@@ -0,0 +1,180 @@
+/**/.gtfs_cache
+/*.local
+.claude
+
+/*gguf*
+
+/docker-config
+/docker_build_local.log
+
+/ms?hit
+*mstest*py
+*_test.py
+/umqtt
+.idea/
+/upydev_.config
+*.local.*
+/scripts/logdir/
+
+
+# Byte-compiled / optimized / DLL files
+__pycache__/
+*.py[cod]
+*$py.class
+
+# C extensions
+*.so
+
+# Distribution / packaging
+.Python
+build/
+develop-eggs/
+dist/
+downloads/
+eggs/
+.eggs/
+lib/
+lib64/
+parts/
+sdist/
+var/
+wheels/
+share/python-wheels/
+*.egg-info/
+.installed.cfg
+*.egg
+MANIFEST
+
+# PyInstaller
+#  Usually these files are written by a python script from a template
+#  before PyInstaller builds the exe, so as to inject date/other infos into it.
+*.manifest
+*.spec
+
+# Installer logs
+pip-log.txt
+pip-delete-this-directory.txt
+
+# Unit test / coverage reports
+htmlcov/
+.tox/
+.nox/
+.coverage
+.coverage.*
+.cache
+nosetests.xml
+coverage.xml
+*.cover
+*.py,cover
+.hypothesis/
+.pytest_cache/
+cover/
+
+# Translations
+*.mo
+*.pot
+
+# Django stuff:
+*.log
+local_settings.py
+db.sqlite3
+db.sqlite3-journal
+
+# Flask stuff:
+instance/
+.webassets-cache
+
+# Scrapy stuff:
+.scrapy
+
+# Sphinx documentation
+docs/_build/
+
+# PyBuilder
+.pybuilder/
+target/
+
+# Jupyter Notebook
+.ipynb_checkpoints
+
+# IPython
+profile_default/
+ipython_config.py
+
+# pyenv
+#   For a library or package, you might want to ignore these files since the code is
+#   intended to run in multiple environments; otherwise, check them in:
+# .python-version
+
+# pipenv
+#   According to pypa/pipenv#598, it is recommended to include Pipfile.lock in version control.
+#   However, in case of collaboration, if having platform-specific dependencies or dependencies
+#   having no cross-platform support, pipenv may install dependencies that don't work, or not
+#   install all needed dependencies.
+#Pipfile.lock
+
+# poetry
+#   Similar to Pipfile.lock, it is generally recommended to include poetry.lock in version control.
+#   This is especially recommended for binary packages to ensure reproducibility, and is more
+#   commonly ignored for libraries.
+#   https://python-poetry.org/docs/basic-usage/#commit-your-poetrylock-file-to-version-control
+#poetry.lock
+
+# pdm
+#   Similar to Pipfile.lock, it is generally recommended to include pdm.lock in version control.
+#pdm.lock
+#   pdm stores project-wide configurations in .pdm.toml, but it is recommended to not include it
+#   in version control.
+#   https://pdm.fming.dev/#use-with-ide
+.pdm.toml
+
+# PEP 582; used by e.g. github.com/David-OConnor/pyflow and github.com/pdm-project/pdm
+__pypackages__/
+
+# Celery stuff
+celerybeat-schedule
+celerybeat.pid
+
+# SageMath parsed files
+*.sage.py
+
+# Environments
+.env
+.venv
+env/
+venv/
+ENV/
+env.bak/
+venv.bak/
+
+# Spyder project settings
+.spyderproject
+.spyproject
+
+# Rope project settings
+.ropeproject
+
+# mkdocs documentation
+/site
+
+# mypy
+.mypy_cache/
+.dmypy.json
+dmypy.json
+
+# Pyre type checker
+.pyre/
+
+# pytype static type analyzer
+.pytype/
+
+# Cython debug symbols
+cython_debug/
+
+# PyCharm
+#  JetBrains specific template is maintained in a separate JetBrains.gitignore that can
+#  be found at https://github.com/github/gitignore/blob/main/Global/JetBrains.gitignore
+#  and can be added to the global gitignore or merged into this file.  For a more nuclear
+#  option (not recommended) you can uncomment the following to ignore the entire idea folder.
+#.idea/
+

somestuff_mqttwebstuff-0.1.4/LICENSE.md

@@ -0,0 +1,163 @@
+GNU Lesser General Public License
+=================================
+
+_Version 3, 29 June 2007_  
+_Copyright © 2007 Free Software Foundation, Inc. &lt;<http://fsf.org/>&gt;_
+
+Everyone is permitted to copy and distribute verbatim copies
+of this license document, but changing it is not allowed.
+
+
+This version of the GNU Lesser General Public License incorporates
+the terms and conditions of version 3 of the GNU General Public
+License, supplemented by the additional permissions listed below.
+
+### 0. Additional Definitions
+
+As used herein, “this License” refers to version 3 of the GNU Lesser
+General Public License, and the “GNU GPL” refers to version 3 of the GNU
+General Public License.
+
+“The Library” refers to a covered work governed by this License,
+other than an Application or a Combined Work as defined below.
+
+An “Application” is any work that makes use of an interface provided
+by the Library, but which is not otherwise based on the Library.
+Defining a subclass of a class defined by the Library is deemed a mode
+of using an interface provided by the Library.
+
+A “Combined Work” is a work produced by combining or linking an
+Application with the Library.  The particular version of the Library
+with which the Combined Work was made is also called the “Linked
+Version”.
+
+The “Minimal Corresponding Source” for a Combined Work means the
+Corresponding Source for the Combined Work, excluding any source code
+for portions of the Combined Work that, considered in isolation, are
+based on the Application, and not on the Linked Version.
+
+The “Corresponding Application Code” for a Combined Work means the
+object code and/or source code for the Application, including any data
+and utility programs needed for reproducing the Combined Work from the
+Application, but excluding the System Libraries of the Combined Work.
+
+### 1. Exception to Section 3 of the GNU GPL
+
+You may convey a covered work under sections 3 and 4 of this License
+without being bound by section 3 of the GNU GPL.
+
+### 2. Conveying Modified Versions
+
+If you modify a copy of the Library, and, in your modifications, a
+facility refers to a function or data to be supplied by an Application
+that uses the facility (other than as an argument passed when the
+facility is invoked), then you may convey a copy of the modified
+version:
+
+* **a)** under this License, provided that you make a good faith effort to
+ensure that, in the event an Application does not supply the
+function or data, the facility still operates, and performs
+whatever part of its purpose remains meaningful, or
+
+* **b)** under the GNU GPL, with none of the additional permissions of
+this License applicable to that copy.
+
+### 3. Object Code Incorporating Material from Library Header Files
+
+The object code form of an Application may incorporate material from
+a header file that is part of the Library.  You may convey such object
+code under terms of your choice, provided that, if the incorporated
+material is not limited to numerical parameters, data structure
+layouts and accessors, or small macros, inline functions and templates
+(ten or fewer lines in length), you do both of the following:
+
+* **a)** Give prominent notice with each copy of the object code that the
+Library is used in it and that the Library and its use are
+covered by this License.
+* **b)** Accompany the object code with a copy of the GNU GPL and this license
+document.
+
+### 4. Combined Works
+
+You may convey a Combined Work under terms of your choice that,
+taken together, effectively do not restrict modification of the
+portions of the Library contained in the Combined Work and reverse
+engineering for debugging such modifications, if you also do each of
+the following:
+
+* **a)** Give prominent notice with each copy of the Combined Work that
+the Library is used in it and that the Library and its use are
+covered by this License.
+
+* **b)** Accompany the Combined Work with a copy of the GNU GPL and this license
+document.
+
+* **c)** For a Combined Work that displays copyright notices during
+execution, include the copyright notice for the Library among
+these notices, as well as a reference directing the user to the
+copies of the GNU GPL and this license document.
+
+* **d)** Do one of the following:
+    - **0)** Convey the Minimal Corresponding Source under the terms of this
+License, and the Corresponding Application Code in a form
+suitable for, and under terms that permit, the user to
+recombine or relink the Application with a modified version of
+the Linked Version to produce a modified Combined Work, in the
+manner specified by section 6 of the GNU GPL for conveying
+Corresponding Source.
+    - **1)** Use a suitable shared library mechanism for linking with the
+Library.  A suitable mechanism is one that **(a)** uses at run time
+a copy of the Library already present on the user's computer
+system, and **(b)** will operate properly with a modified version
+of the Library that is interface-compatible with the Linked
+Version.
+
+* **e)** Provide Installation Information, but only if you would otherwise
+be required to provide such information under section 6 of the
+GNU GPL, and only to the extent that such information is
+necessary to install and execute a modified version of the
+Combined Work produced by recombining or relinking the
+Application with a modified version of the Linked Version. (If
+you use option **4d0**, the Installation Information must accompany
+the Minimal Corresponding Source and Corresponding Application
+Code. If you use option **4d1**, you must provide the Installation
+Information in the manner specified by section 6 of the GNU GPL
+for conveying Corresponding Source.)
+
+### 5. Combined Libraries
+
+You may place library facilities that are a work based on the
+Library side by side in a single library together with other library
+facilities that are not Applications and are not covered by this
+License, and convey such a combined library under terms of your
+choice, if you do both of the following:
+
+* **a)** Accompany the combined library with a copy of the same work based
+on the Library, uncombined with any other library facilities,
+conveyed under the terms of this License.
+* **b)** Give prominent notice with the combined library that part of it
+is a work based on the Library, and explaining where to find the
+accompanying uncombined form of the same work.
+
+### 6. Revised Versions of the GNU Lesser General Public License
+
+The Free Software Foundation may publish revised and/or new versions
+of the GNU Lesser General Public License from time to time. Such new
+versions will be similar in spirit to the present version, but may
+differ in detail to address new problems or concerns.
+
+Each version is given a distinguishing version number. If the
+Library as you received it specifies that a certain numbered version
+of the GNU Lesser General Public License “or any later version”
+applies to it, you have the option of following the terms and
+conditions either of that published version or of any later version
+published by the Free Software Foundation. If the Library as you
+received it does not specify a version number of the GNU Lesser
+General Public License, you may choose any version of the GNU Lesser
+General Public License ever published by the Free Software Foundation.
+
+If the Library as you received it specifies that a proxy can decide
+whether future versions of the GNU Lesser General Public License shall
+apply, that proxy's public statement of acceptance of any version is
+permanent authorization for you to choose that version for the
+Library.

somestuff_mqttwebstuff-0.1.4/PKG-INFO

@@ -0,0 +1,230 @@
+Metadata-Version: 2.5
+Name: somestuff-mqttwebstuff
+Version: 0.1.4
+Summary: Live web view onto MQTT streams: FastAPI + mountable mapper plugins, Server-Sent Events, htmx/PicoCSS frontend
+Project-URL: Homepage, https://github.com/vroomfondel/somestuff
+Project-URL: Repository, https://github.com/vroomfondel/somestuff
+Project-URL: Issues, https://github.com/vroomfondel/somestuff/issues
+Author-email: Henning Thieß <ht@xomox.cc>
+License-File: LICENSE.md
+Keywords: dashboard,fastapi,htmx,iot,mqtt,server-sent-events,sse
+Classifier: Development Status :: 4 - Beta
+Classifier: Framework :: FastAPI
+Classifier: Intended Audience :: System Administrators
+Classifier: License :: OSI Approved :: GNU Lesser General Public License v3 (LGPLv3)
+Classifier: Operating System :: POSIX :: Linux
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.13
+Classifier: Programming Language :: Python :: 3.14
+Classifier: Topic :: Home Automation
+Classifier: Topic :: Internet :: WWW/HTTP :: HTTP Servers
+Classifier: Typing :: Typed
+Requires-Python: >=3.13
+Requires-Dist: fastapi>=0.143
+Requires-Dist: jinja2>=3.1
+Requires-Dist: loguru>=0.7.3
+Requires-Dist: mqttstuff>=0.0.7
+Requires-Dist: python-dotenv>=1.2
+Requires-Dist: tabulate>=0.10
+Requires-Dist: typer>=0.27
+Requires-Dist: uvicorn>=0.54
+Description-Content-Type: text/markdown
+
+# mqttwebstuff
+
+A tiny live web view onto arbitrary MQTT streams: FastAPI subscribes to the
+broker, a **mapper plugin** (a plain, mountable Python file or module)
+filters/maps every message, and the browser receives rendered HTML fragments via
+**Server-Sent Events**, displayed with **htmx** (+ SSE extension) on **PicoCSS**.
+No frontend build, no framework; the only hand-written JavaScript is a dozen
+lines in `base.html.j2` preserving open `<details>` states across SSE swaps.
+
+## Modules
+
+- **`serve.py`**: Typer CLI (`mqttweb-serve`). Loads the mapper plugin (or the
+  generic fallback), connects to the broker and runs the web app with uvicorn.
+  The MQTT connection tolerates an unreachable broker at startup: paho keeps
+  retrying in the background while the web app is already up (empty board,
+  `/healthz` showing `mqtt_connected: false`).
+- **`hub.py`**: `ViewHub`, the last-value cache between MQTT and the browsers.
+  It keeps the last rendered item per `(panel, key)` and fans updates out to all
+  SSE clients. Messages are submitted thread-safely from paho's network thread.
+- **`plugin_api.py`**: the contract between core and plugin. `ViewEvent`
+  describes one mapped message (panel, key, data, template, sort, group, title,
+  ttl); `load_plugin` validates a mapper given as a file path or dotted module.
+- **`webapp.py`**: the FastAPI app and the Jinja2 environment (including the
+  `hhmm` filter).
+- **`plugins/oepnv_view.py`**: reference mapper rendering the `oepnvstuff`
+  GTFS-realtime stream as a departure board (departures, feed status,
+  per-line coverage, service alerts). It follows the publisher's topic base via
+  `OEPNV_MQTT_BASE_TOPIC` (default `oepnv`; multi-level bases like `home/oepnv`
+  work).
+- **`templates/`, `static/`**: built-in Jinja2 templates (generic JSON card,
+  indented topic tree) and vendored htmx, htmx-ext-sse and PicoCSS, so the
+  server serves everything itself (no CDN egress).
+
+Logging (loguru, with an intercept for stdlib `logging` from `paho` and
+`uvicorn`) and the startup banner are shared via `somestuff.mqttwebstuff`.
+
+```
+MQTT broker ──> paho thread ──> ViewHub (last-value cache) ──> SSE ──> htmx swaps panels
+                                    │
+                              mapper plugin
+                              filter / panel / key / template / ttl
+```
+
+### Why a cache in the middle?
+
+Streams such as the oepnv one publish with `retain=False`. The hub keeps the
+last rendered item per `(panel, key)`, so a freshly opened tab gets the full
+current board server-rendered on `GET /`, then only deltas over `GET /stream`.
+Items carry a TTL, so topics that stop being published disappear from the
+board, mirroring the non-retained semantics.
+
+### Endpoints
+
+| Route      | Purpose                                                         |
+|------------|-----------------------------------------------------------------|
+| `/`        | Full board, server-rendered from the cache                      |
+| `/stream`  | SSE: `panel:<name>` repaints, `board` on new panels, keepalives |
+| `/healthz` | `{"ok": true, "mqtt_connected": ...}`                           |
+
+### Deep links
+
+Group boxes (e.g. one stop on the oepnv board) and tree rows carry stable,
+ASCII-slugged anchor ids: `/#g-<panel>-<label>` and `/#t-<topic>`, e.g.
+`/#g-departures-ellerbek-moordamm` or `/#t-nodered-hydrostatics-hydrostatic1`.
+Hovering a label reveals its self-link (`#`); the jump target is highlighted.
+Since ids survive repaints and restarts, the links are bookmarkable.
+
+## Installation
+
+```bash
+pip install somestuff-mqttwebstuff
+```
+
+`somestuff` is a PEP 420 namespace; the package is imported as
+`somestuff.mqttwebstuff` and shares the namespace with the other `somestuff-*`
+distributions.
+
+## Usage
+
+The console script and module invocation are equivalent:
+
+```bash
+mqttweb-serve --mapper somestuff.mqttwebstuff.plugins.oepnv_view --mqtt-host broker.example.org
+python3 -m somestuff.mqttwebstuff.serve --mapper somestuff.mqttwebstuff.plugins.oepnv_view --mqtt-host broker.example.org
+```
+
+`--mapper` accepts a dotted module name on the import path (as above) or the
+path of a `.py` file, e.g. `--mapper /config/my_view.py`. In a repository
+checkout, `--mapper somestuff/mqttwebstuff/plugins/oepnv_view.py` works as well.
+
+Without a mapper, a generic view shows every message of the `--topics` patterns:
+
+```bash
+# default layout: indented hierarchical topic tree (branch rows per level,
+# scalars inline, JSON collapsible)
+mqttweb-serve --topics 'ecowitt/#,oepnv/status' --mqtt-host broker.example.org
+
+# ad-hoc: peek into a whole topic tree and keep items forever
+mqttweb-serve --mapper "" --topics 'nodered/#' --item-ttl 0 --title "Node-RED Live"
+
+# one JSON card per topic instead of the tree
+mqttweb-serve --mapper "" --topics 'nodered/#' --view flat
+```
+
+`--mapper ""` explicitly overrides a mapper configured via environment or env
+file; a configured mapper otherwise wins over `--topics`. `--item-ttl` defaults
+to 900 seconds (an item without a new message disappears), `0` keeps items
+forever. `mqttweb-serve --help` lists all options.
+
+## Configuration
+
+Every option is also an environment variable (CLI wins over environment). The
+tool additionally loads `mqttweb.local.env` from the current working directory
+and from the package directory; precedence is real environment, then the CWD
+file, then the package-directory file, with CLI options over all.
+
+| Variable                    | CLI option            | Default                                 | Meaning                                                                                                                                                                           |
+|-----------------------------|-----------------------|-----------------------------------------|-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
+| `MQTTWEB_MAPPER`            | `--mapper`            | empty (generic view)                    | Mapper plugin: `.py` file path or dotted module name                                                                                                                              |
+| `MQTTWEB_TOPICS`            | `--topics`            | `#`                                     | Comma-separated subscribe patterns, generic view only                                                                                                                             |
+| `MQTTWEB_ITEM_TTL`          | `--item-ttl`          | `900`                                   | Generic view: seconds until an unrefreshed card disappears, `0` = forever                                                                                                         |
+| `MQTTWEB_VIEW`              | `--view`              | `hierarchical`                          | Generic view layout: `hierarchical` or `flat`                                                                                                                                     |
+| `MQTTWEB_TITLE`             | `--title`             | empty (plugin's `TITLE`)                | Page title override                                                                                                                                                               |
+| `MQTTWEB_LISTEN_HOST`       | `--listen-host`       | `0.0.0.0`                               | Bind address                                                                                                                                                                      |
+| `MQTTWEB_LISTEN_PORT`       | `--listen-port`       | `8080`                                  | Bind port                                                                                                                                                                         |
+| `MQTTWEB_LIMIT_CONCURRENCY` | `--limit-concurrency` | `0` (unlimited)                         | Reject new connections with 503 above N concurrent ones. Every open tab holds one SSE stream and a page load briefly needs several slots, so size generously (100+), not per user |
+| `MQTTWEB_MQTT_HOST`         | `--mqtt-host`         | `mosquitto.mosquitto.svc.cluster.local` | MQTT broker host                                                                                                                                                                  |
+| `MQTTWEB_MQTT_PORT`         | `--mqtt-port`         | `1883`                                  | MQTT broker port                                                                                                                                                                  |
+| `MQTTWEB_MQTT_USER`         | `--mqtt-user`         | empty (anonymous)                       | MQTT username                                                                                                                                                                     |
+| `MQTTWEB_MQTT_PASSWORD`     | `--mqtt-password`     | empty                                   | MQTT password                                                                                                                                                                     |
+| `MQTTWEB_MQTT_TLS`          | `--mqtt-tls`          | `false`                                 | Encrypt the MQTT connection; the port is not switched automatically, set `--mqtt-port` (usually 8883)                                                                             |
+| `MQTTWEB_MQTT_TLS_CA`       | `--mqtt-tls-ca`       | empty (system CA store)                 | CA certificate path                                                                                                                                                               |
+| `MQTTWEB_MQTT_TLS_CERT`     | `--mqtt-tls-cert`     | empty                                   | Client certificate path (mutual TLS)                                                                                                                                              |
+| `MQTTWEB_MQTT_TLS_KEY`      | `--mqtt-tls-key`      | empty                                   | Client key path (mutual TLS)                                                                                                                                                      |
+| `MQTTWEB_MQTT_TLS_INSECURE` | `--mqtt-tls-insecure` | `false`                                 | Skip TLS hostname verification (self-signed certs); encrypted but MITM-able                                                                                                       |
+| `MQTTWEB_VERBOSE`           | `--verbose`, `-v`     | `false`                                 | DEBUG logging, including the uvicorn access log                                                                                                                                   |
+
+`--topics`, `--item-ttl` and `--view` are ignored when a mapper is set.
+Plugin-specific settings are plain environment variables the plugin reads
+itself; the oepnv plugin reads `OEPNV_MQTT_BASE_TOPIC` at load time.
+
+The repository additionally contains an annotated `mqttweb.env.example`, a
+Kubernetes deployment example and a secret example (not part of the installed
+package): https://github.com/vroomfondel/somestuff/tree/main/somestuff/mqttwebstuff/
+
+## Writing a mapper plugin
+
+A plugin is one Python file (or an importable module), no packaging needed.
+Required attributes: `SUBSCRIPTIONS` and `map_message`. Optional: `TITLE`,
+`PANELS` (panel name to heading, fixes the board order; an empty heading means a
+plain panel without chrome) and `TEMPLATE_DIR`. `payload` is already decoded: a
+`dict`/`list` when the raw payload parses as JSON, otherwise the raw string.
+
+### Library
+
+```python
+from somestuff.mqttwebstuff.plugin_api import ViewEvent
+
+TITLE = "My Stream"
+SUBSCRIPTIONS = ["mystream/#"]
+PANELS = {"main": "Main"}          # optional: fixes panel order + headings
+
+# May also return a list of ViewEvents (one message -> several board items,
+# e.g. a leaf plus synthesized ancestor rows, as the generic hierarchical view
+# does) or None to drop the message.
+def map_message(topic: str, payload) -> ViewEvent | None:
+    if not isinstance(payload, dict):
+        return None                # filter
+    return ViewEvent(
+        panel="main",              # board section
+        key=topic,                 # stable identity -> idempotent replace
+        data=payload,              # template context
+        template=None,             # None = built-in generic JSON card
+        sort="",                   # ordering within the panel/group ("" = by key)
+        group="",                  # optional sub-heading label (e.g. stop name);
+                                   # items cluster under it, "" = ungrouped
+        ttl=300.0,                 # vanish after 5 min without re-publish
+    )
+```
+
+Save it as `my_view.py` and run `mqttweb-serve --mapper my_view.py --mqtt-host broker.example.org`.
+
+Templates: put Jinja2 files (`*.html.j2`) into a `templates/` directory next to
+the plugin file (or set `TEMPLATE_DIR`, relative to the plugin's directory). The
+plugin's directory is searched first, the built-ins second, so a plugin can also
+override `base.html.j2` or add styles via an `extra_head.html.j2` include. The
+`hhmm` filter formats ISO timestamps as `HH:MM`.
+
+In Kubernetes, mount the plugin (+ templates) from a ConfigMap and point
+`MQTTWEB_MAPPER` at it; see the deployment example in the repository (link
+above). The oepnv reference plugin is part of the package, so the oepnv board
+needs no mount at all.
+
+## Notes
+
+- Requires: `loguru`, `typer`, `fastapi`, `uvicorn`, `jinja2`, `python-dotenv`,
+  `tabulate`, `mqttstuff`.
+- Licensed under the LGPL, see `LICENSE.md` in the repository root.