unifioptimizer 0.1.0__tar.gz

unifioptimizer-0.1.0/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 Gary Neitzke
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

unifioptimizer-0.1.0/MANIFEST.in

@@ -0,0 +1,18 @@
+# Source-distribution contents. The wheel's package data is declared in
+# pyproject.toml ([tool.setuptools.package-data]); this file makes `python -m
+# build` (which builds the wheel FROM the sdist) also carry the prebuilt web UI,
+# so the bundled dashboard survives the sdist -> wheel round-trip.
+#
+# netadmin/_webui/ is a build artifact (produced by `python tools/build_web.py`,
+# not tracked in git), so it is grafted here explicitly rather than via VCS.
+graft netadmin/_webui
+
+# Numbered SQL migrations applied at store startup — runtime data, must ship.
+recursive-include netadmin/store/migrations *.sql
+
+include README.md
+include LICENSE
+include requirements.txt
+
+# Keep bytecode / caches out of the distribution.
+global-exclude __pycache__ *.py[cod] *.so .DS_Store

unifioptimizer-0.1.0/PKG-INFO

@@ -0,0 +1,483 @@
+Metadata-Version: 2.4
+Name: unifioptimizer
+Version: 0.1.0
+Summary: A network admin that remembers: a stateful UniFi monitor with a local time-series store, confounder-aware detectors, and an issue-lifecycle engine.
+Author-email: Gary Neitzke <gneitzke123@gmail.com>
+License-Expression: MIT
+Project-URL: Homepage, https://github.com/gneitzke/UnifiOptimizer
+Project-URL: Repository, https://github.com/gneitzke/UnifiOptimizer
+Project-URL: Issues, https://github.com/gneitzke/UnifiOptimizer/issues
+Project-URL: Documentation, https://github.com/gneitzke/UnifiOptimizer/blob/main/docs/ARCHITECTURE.md
+Keywords: unifi,ubiquiti,network-monitoring,wifi,self-hosted,sqlite,home-assistant,observability
+Classifier: Development Status :: 4 - Beta
+Classifier: Environment :: Console
+Classifier: Environment :: Web Environment
+Classifier: Framework :: FastAPI
+Classifier: Intended Audience :: System Administrators
+Classifier: Operating System :: MacOS
+Classifier: Operating System :: POSIX :: Linux
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Topic :: System :: Networking :: Monitoring
+Classifier: Topic :: System :: Systems Administration
+Requires-Python: >=3.11
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Requires-Dist: httpx>=0.27
+Requires-Dist: pydantic>=2
+Requires-Dist: pydantic-settings>=2
+Requires-Dist: APScheduler>=3.10
+Requires-Dist: websockets>=12
+Requires-Dist: pyyaml>=6
+Requires-Dist: rich>=13
+Requires-Dist: fastapi>=0.104
+Requires-Dist: uvicorn[standard]>=0.24
+Requires-Dist: dnspython>=2
+Requires-Dist: aiomqtt>=2
+Provides-Extra: test
+Requires-Dist: pytest>=8; extra == "test"
+Requires-Dist: pytest-asyncio>=0.23; extra == "test"
+Requires-Dist: respx>=0.21; extra == "test"
+Dynamic: license-file
+
+# UnifiOptimizer
+
+**A network admin that remembers.** UnifiOptimizer watches a UniFi network the way a
+good technician would: it keeps a running history, notices when something starts
+misbehaving, waits to rule out the obvious false alarms, tells you what it thinks
+is wrong and why, and then checks whether the fix actually held.
+
+This is a ground-up rebuild of UnifiOptimizer. The original tool ran a stateless
+snapshot on demand and printed a report. UnifiOptimizer keeps state. That one change is
+the whole point: a controller throws away its fine-grained stats after about a
+day, so anything you do not collect daily is simply gone, and no snapshot tool can
+tell you that a port started erroring on Tuesday or that a mesh link has been
+sliding for a week. UnifiOptimizer can, because it was watching.
+
+The rebuild lives in the `netadmin/` Python package and is the whole project now;
+the original stateless `optimizer.py` CLI was removed at cutover.
+
+Architecture and design decisions are documented in full at
+[`docs/ARCHITECTURE.md`](docs/ARCHITECTURE.md), and there is a plain-language,
+hand-drawn walkthrough at [`docs/HOW_IT_WORKS.md`](docs/HOW_IT_WORKS.md).
+
+![UnifiOptimizer dashboard: an 87/100 health score, a collector-health strip, six service-level cards with 24-hour trend sparklines, active incidents grouped by severity, and an Export report action.](docs/img/screenshots/dashboard-dark.png)
+
+## Quick start
+
+```bash
+pip install unifioptimizer
+
+# see it working on a fictional, PII-free demo network (no controller needed)
+netadmin demo-seed --out data/netadmin-demo.db --now $(( $(date +%s) / 300 * 300 ))
+NETADMIN_DB_PATH=data/netadmin-demo.db netadmin daemon   # then open http://localhost:8765
+
+# or point it at your own controller: put credentials in data/secrets.env (below)
+netadmin daemon
+```
+
+The wheel bundles the compiled dashboard, so `pip install` alone gives you a
+working UI with no Node.js. Running from a source checkout instead? See
+[Install](#install) — it needs one build step.
+
+---
+
+## What it does
+
+UnifiOptimizer is one Python process that runs a collector, a detection engine, an
+issue-lifecycle tracker, a health model, and a small web/API server on the same
+event loop.
+
+- **Collects history into a local store.** Every 60 seconds it pulls device,
+  client, and health stats over the controller's REST API, listens to the event
+  WebSocket in real time, and runs its own DNS and ICMP probes for the timing the
+  controller does not report. Everything lands in one SQLite file (`data/netadmin.db`,
+  WAL mode). On a small home network that file sits around 20 MB after a day with
+  a few hundred thousand samples. Counters are stored as per-interval rates, gaps
+  are recorded rather than papered over with zeros, and old data rolls up hourly
+  then daily so year-over-year comparisons stay possible.
+
+- **Detects problems, with the false-alarm checks written down.** Detectors are
+  deterministic rules, not a black box: static thresholds plus rolling
+  quantile bands off each series' own baseline. A cable detector that fires on a
+  gigabit port stuck at 100 Mbps first confirms the port is genuinely
+  gigabit-capable and the attached device is not a known 100 Mbps class, and it
+  records those checks alongside the finding. That audit trail is what separates
+  an admin from an alarm generator. The catalog covers wired faults (bad cable,
+  duplex, port flapping, PoE budget, STP loops, SFP degradation), WiFi
+  (sticky clients, ping-pong roamers, channel plan, DFS, airtime saturation,
+  mesh backhaul), clients (flaky disconnects with reason-code weighting, DHCP
+  failures), and WAN (ISP degradation, bufferbloat, DNS slowness, WAN flapping).
+
+- **Tracks each issue's whole life.** A finding does not become a fresh alert
+  every poll. It gets a fingerprint, and one open issue exists per fingerprint.
+  An issue moves `pending -> active -> resolving -> resolved`, carries a
+  "still occurring, day 5" clock, reopens the same row if it refires within a day
+  instead of spawning a duplicate, and gets suppressed when a bigger fault
+  explains it (a downed switch mutes its own ports' issues). Every state change
+  is logged, so nothing about an issue is untraceable.
+
+- **Scores health honestly (Mist-style SLE).** Each five-minute bucket, each
+  active client contributes minutes judged pass or fail per service-level
+  expectation (coverage, roaming, capacity, connect, WAN, infrastructure). Every
+  failed minute is pinned to exactly one cause on one device. The headline health
+  number and its explanation are the same query, so "88.6% overall, coverage
+  84%, 138 failed client-minutes, top offender the Living Room AP" is one click from
+  the score. An idle client with bad signal contributes zero failed minutes,
+  which is what keeps the number impact-weighted instead of theatrical.
+
+- **Proposes fixes you approve.** When a fix maps cleanly to a controller config
+  change (channel plan, transmit power step-down, removing a misapplied min-RSSI,
+  cycling a PoE port), UnifiOptimizer can render the exact API payload, show you a
+  full before-state, and apply it only when you click. It then watches for a
+  verification window to confirm the issue actually cleared. Nothing applies on
+  its own. See [Safety model](#safety-model).
+
+- **Explains, when you want a second opinion.** For any issue, UnifiOptimizer compiles
+  a markdown dossier: the issue trail, the evidence windows as compact tables,
+  related issues on the same segment, the confounders already ruled out, and the
+  relevant playbook entry. You can hand that to any model yourself (the default,
+  no API key needed), pipe it through GitHub Copilot CLI, or wire an Anthropic
+  key. The investigator explains and correlates; it never applies anything.
+
+- **Alerts through Home Assistant.** Optional and off by default. Over MQTT
+  discovery it publishes a health sensor, per-severity issue counts, and a
+  binary sensor per active P1/P2 that clears on resolve, so HA automations can
+  notify you on a new critical issue.
+
+---
+
+## How it works
+
+The one idea behind UnifiOptimizer is memory. A controller throws away its
+fine-grained stats after about a day, so UnifiOptimizer keeps its own history and reads
+everything else (detection, health, issue tracking) from that. The full
+walkthrough is in [`docs/HOW_IT_WORKS.md`](docs/HOW_IT_WORKS.md); the short of it:
+
+**It watches, remembers, then tells you.**
+
+![How UnifiOptimizer watches: the UniFi controller feeds a 60-second collector and DNS/ICMP probes; everything lands in one SQLite store that rolls up hourly then daily; detectors and the health model read from it and feed the issue engine, which reaches you over web and Home Assistant. Fixes loop back to the controller only on your approval.](docs/img/how-netadmin-watches.png)
+
+**One issue per fingerprint, not a new alert every poll.**
+
+![The life of an issue: many repeated findings collapse into one fingerprint that moves through pending, active, resolving, and resolved. A refire within a day reopens the same row, a fire during resolving snaps back to active, and a bigger fault mutes the smaller ones it explains.](docs/img/life-of-an-issue.png)
+
+**Health you can argue with.**
+
+![Health as user-minutes: each active client-minute is judged pass or fail, each failed minute is pinned to one cause on one device, and an idle client with bad signal contributes zero failed minutes. The score and its explanation are the same query.](docs/img/health-sle.png)
+
+---
+
+## The interface
+
+The web UI ships with the daemon and renders in light and dark. Every screenshot
+here is from the built-in `netadmin demo-seed` network: fictional devices,
+fabricated MACs, documentation-range IPs, so nothing below is a real network.
+
+The issues view is the triage surface: every open finding with its severity, the
+detector that raised it, the affected device, and how long it has been going.
+
+![The issues list: fourteen findings ranked by severity, each with its state (active or resolving), the detector that raised it (port flapping, weak mesh backhaul, DNS slow, rogue AP, sticky client), the affected entity, and its duration.](docs/img/screenshots/issues-light.png)
+
+Open one and it carries its whole lifecycle: the evidence, the false alarms ruled
+out, and a proposed fix you approve before anything touches the controller.
+
+![An issue detail page showing a resolved channel-plan issue with its evidence, the confounders that were ruled out, the lifecycle trail, and a fix that was proposed, applied, and verified.](docs/img/screenshots/issue-fix-light.png)
+
+When you need to hand someone the whole picture, **Export report** renders a
+print-ready network assessment: executive summary, topology, per-service health,
+RF and client analysis, and a walkthrough of every finding. It states its own data
+window and poll coverage up front, and every number traces to a stored query.
+There is no sample or projected data.
+
+![The exported assessment's executive summary: an overall health score, findings counted by severity, and the highest-impact findings in plain language with the client-hours each one cost.](docs/img/screenshots/report-exec-light.png)
+
+First run points UnifiOptimizer at your console. Type the address, or let it scan
+the network for you, and the API key is written to the daemon, never shown in the
+browser or sent anywhere else.
+
+![The UnifiOptimizer first-run screen in dark mode: a "Connect your network" heading, a controller-address field with a "Scan my network" option, a "Detect" button, and a note that the API key is written only to the daemon.](docs/img/screenshots/setup-dark.png)
+
+---
+
+## Two ways to run it
+
+Both modes share every layer. The on-demand mode is literally the daemon's
+startup path without the scheduler.
+
+### Daemon (always-on)
+
+The permanent home. It backfills whatever the controller still retains, then
+polls, detects, tracks, and scores continuously.
+
+```bash
+netadmin daemon                 # binds 127.0.0.1:8765 by default
+netadmin status                 # hit a running daemon's /api/health
+netadmin status --json          # ...and print the raw health payload
+```
+
+Healthy status shows `status: ok`, collector jobs green with resetting poll
+ages, `websocket.state: running`, and `backfill: done`.
+
+### Tech visit (on demand)
+
+One pass over the history the controller still holds, then exit. This is the
+"walk in, look around, leave a report" mode for a network you are not running
+the daemon against.
+
+```bash
+netadmin visit --lookback-days 3   # backfill + detect + report over a 3-day window
+```
+
+The daemon is the recommended mode. A visit can only analyze what the controller
+still retains (roughly a day of five-minute stats), which is exactly the gap the
+daemon exists to close.
+
+---
+
+## Install
+
+**Requirements**
+
+- **Python 3.11+**
+- A **UniFi controller** (CloudKey Gen2/Gen2+, UDM/UDM-Pro, or self-hosted
+  Network application) reachable on your LAN
+- An **admin** account on the controller, or an **API key** (UniFi OS consoles
+  on Network 9.x). Read-only controller accounts do not expose the stats and
+  events UnifiOptimizer needs.
+- **Node.js 18+** only to build the web UI from a source checkout; the published
+  wheel ships it prebuilt, so `pip install unifioptimizer` needs no Node.js
+
+**Get a credential.** On a modern UniFi OS console, create a revocable API key
+(Settings -> Control Plane -> Integrations); on an older or self-hosted
+controller, use a dedicated local admin account instead. Ubiquiti moves that
+screen between firmware versions, so rather than guess, run
+
+```bash
+netadmin detect --host YOUR-CONTROLLER
+```
+
+and it reads your console and prints the exact path to click for your device.
+The full per-console walkthrough, the version requirements, and what to put in
+`data/secrets.env` are in [`docs/CONTROLLER_SETUP.md`](docs/CONTROLLER_SETUP.md).
+
+**Install the package**
+
+The published wheel ships the compiled dashboard inside it, so end users need no
+Node.js:
+
+```bash
+pip install unifioptimizer
+```
+
+From a source checkout, build the dashboard once so the daemon can serve it:
+
+```bash
+git clone https://github.com/gneitzke/UnifiOptimizer.git
+cd UnifiOptimizer
+pip install -e .                 # console script + runtime deps
+python tools/build_web.py        # compile + bundle the dashboard (needs Node 18+)
+```
+
+`./install.sh` runs both steps (and creates a venv) in one command. Skipping the
+build leaves the API and daemon fully working; only the web dashboard waits until
+you build it. Either way you get the runtime deps: httpx, pydantic, APScheduler,
+websockets, FastAPI/uvicorn, dnspython, aiomqtt.
+
+---
+
+## Configuration
+
+Two files under `data/`, both read at runtime, neither ever committed:
+
+- **`data/secrets.env`** (chmod 600, gitignored) holds credentials:
+
+  ```ini
+  UNIFI_HOST=https://192.168.1.1
+  UNIFI_API_KEY=your-api-key            # preferred; or the pair below
+  # UNIFI_USERNAME=audit
+  # UNIFI_PASSWORD=...
+  UNIFI_SITE=default
+  ```
+
+- **`data/config.yaml`**, under a `netadmin:` block, holds structural config:
+  the SQLite path, the API bind (`server_host`/`server_port`, default
+  `127.0.0.1:8765`), pinned CORS origins, collector cadences, retention tiers,
+  probe targets, and the Home Assistant block. Every key is optional and falls
+  back to the defaults in `netadmin/config.py`. Two settings are worth checking
+  on first run:
+
+  ```yaml
+  netadmin:
+    probe:
+      gateway_ip: 192.168.1.1      # DNS/RTT probe target; auto-discovered if unset
+      anchor: 1.1.1.1              # public resolver, the "is it me or my ISP" comparison
+    wan_plan_down_mbps: null       # set your plan rate to enable WAN saturation
+    wan_plan_up_mbps: null         # and bufferbloat detection (null = those detectors abstain)
+  ```
+
+Environment variables override YAML: `DB_PATH`, `LOG_LEVEL`, `SITE_ID`,
+`SERVER_HOST`, `SERVER_PORT` map directly to the field names (no prefix).
+
+---
+
+## Safety model
+
+**Read-only by default.** The current build talks to the controller with GETs
+and a small set of documented read-query POSTs. There is no automatic mutation
+anywhere in the collector or the daemon.
+
+When the fix engine is enabled, it holds to hard rules:
+
+- **The daemon never applies on its own.** Every apply is a deliberate human
+  action, a CLI flag or a UI button. There is no scheduler job or callback that
+  applies a fix.
+- **Dry-run is the default.** A dry-run renders the exact API payload without
+  sending it. Applying requires an explicit confirm flag.
+- **Configuration changes are revertible.** An apply captures the full
+  before-state, records the change to the store, and reverts in one click. (A PoE
+  power-cycle is momentary and has no state to restore; it is marked as such.)
+- **Blast radius is capped.** No more than a configured number of devices change
+  per apply, and mesh-uplink APs' min-RSSI is never touched except to remove a
+  misapplied one.
+
+From the CLI a fix is a dry-run by default and applies only with an explicit
+confirm:
+
+```bash
+netadmin fix 42                    # render the exact payload; send nothing
+netadmin fix 42 --apply --confirm  # apply it, after capturing the before-state
+netadmin fix --revert 7            # restore change 7 from its saved before-state
+```
+
+In the web UI the same plan appears under Proposed fix on the issue page: an
+Apply button behind a confirmation modal, and Revert on any change already
+applied. The confirm token binds each apply to the exact plan you reviewed, so a
+device that changed since you looked is refused rather than applied blind.
+
+The daemon's HTTP API has no authentication yet, so it binds loopback only. Do
+not publish port 8765 to an untrusted network without your own authenticating
+proxy in front of it; reach a remote daemon over an SSH tunnel. Full policy in
+[`SECURITY.md`](SECURITY.md).
+
+---
+
+## What it can and cannot do
+
+Honesty about limits is part of the design.
+
+- **Physical faults it flags but cannot fix.** A bad cable, a mesh AP with a
+  −81 dBm backhaul, a coverage hole with no better AP for the affected clients:
+  UnifiOptimizer identifies these with evidence and tells you where to look, but the
+  fix is your hands on hardware. The fix engine only changes controller config.
+
+- **WAN detection needs your plan rate, and Starlink is noisy.** Saturation and
+  bufferbloat detectors compare throughput against your configured plan rate;
+  leave `wan_plan_*_mbps` null and they abstain rather than guess. Starlink and
+  other variable-rate or CGNAT links make "plan rate" fuzzy and the WAN latency
+  baseline drift, so those detectors lean on trend over absolute thresholds and
+  will be less confident on such links than on a fixed-rate connection.
+
+- **Controller version changes what is available.** On Network 9.x the `stat/event`
+  history endpoint was removed; events now come only from the live WebSocket,
+  which means no historical event backlog before the daemon first started
+  watching. Unofficial v2 endpoints are probed at startup and used only when
+  present. When evidence is thin, detectors return UNKNOWN instead of guessing.
+
+- **Some things are out of scope, stated plainly.** No late-collision detection
+  (no counter exposed), no ARP-conflict visibility, no confident non-WiFi
+  interferer identification or hidden-node detection, no client-side downlink
+  RSSI. UnifiOptimizer infers "unexplained airtime utilization" but will not name the
+  microwave.
+
+- **PoE port power-cycle is not revertible.** Cycling power to a port is a
+  momentary action with no before-state to restore, so that one fix template is
+  marked non-revertible. Every configuration change (channel, power, min-RSSI)
+  captures its full before-state and reverts in one click.
+
+---
+
+## Deployment
+
+The daemon image, `Dockerfile.netadmin`, is a single arch-neutral Python image
+that builds unchanged for both `linux/arm64` and `linux/amd64`. Every dependency
+ships prebuilt wheels for both, and nothing shells out to an arch-specific binary.
+There are two documented build paths, each producing the same image:
+
+- **amd64 / x86 NAS or server, with `docker buildx`.**
+
+  ```bash
+  ./deploy/build-multiarch.sh          # builds arm64 + amd64, inspects, verifies
+  ```
+
+  The script builds both platforms to a local OCI tarball, prints the manifest
+  list to prove both are present, and pushes nothing. Set `SMOKE=1` to also load
+  the host-native arch and run an import test. Publish to a registry only when you
+  have one and want to; the script prints the exact `--push` command.
+
+- **arm64 / Apple Silicon, with Apple `container`.** On macOS, Apple's `container`
+  CLI builds and runs the same image from a git-archive tarball with no secrets in
+  it. Bind the daemon to loopback, hand off the SQLite path, and keep it up with a
+  LaunchAgent; the API is unauthenticated, so never expose it on the LAN.
+
+Whichever way you build it, the container holds no secrets. Credentials and the
+SQLite store live on a bind-mounted `data/` volume, and `.dockerignore` keeps
+`secrets.env` and the database out of every image layer.
+
+---
+
+## Coming from the old tool
+
+The original `optimizer.py` analyze/optimize CLI has been removed. Its
+one-shot analysis is now the daemon's continuous job, its HTML report became the
+web UI and the on-demand tech visit, and its change-application became the
+approval-gated fix engine. There is nothing to migrate: point UnifiOptimizer at your
+controller and it starts building the history the old tool never kept.
+
+---
+
+## Testing
+
+```bash
+python -m pytest tests/netadmin -q       # the rebuild's suite
+pip install -e ".[test]"                  # pytest, pytest-asyncio, respx
+```
+
+The pure-logic layers (issue engine, baselines, SLE, detectors) have exhaustive
+unit tests including confounder cases, controller payloads are replayed from
+sanitized recorded fixtures, and an end-to-end test drives a synthetic bad week
+(a cable degrades, a client flaps, firmware regresses) through ingest, detection,
+and the issue lifecycle. No test ever touches a live controller; every mutating
+path is exercised against mocks and dry-run rendering only.
+
+---
+
+## Documentation
+
+- [`docs/ARCHITECTURE.md`](docs/ARCHITECTURE.md): the full design. Data model,
+  detector catalog, issue engine, SLE model, fix engine, the whole spine.
+- [`docs/HOW_IT_WORKS.md`](docs/HOW_IT_WORKS.md): a plain-language, hand-drawn
+  walkthrough of the whole thing.
+- [`docs/DESIGN_FOUNDATION.md`](docs/DESIGN_FOUNDATION.md): the web UI design
+  contract.
+- [`docs/CONTROLLER_SETUP.md`](docs/CONTROLLER_SETUP.md): per-console API-key setup
+  (run `netadmin detect` and it tells you where to click for your device).
+- [`docs/BACKUP.md`](docs/BACKUP.md): backing up and restoring the database.
+- [`SECURITY.md`](SECURITY.md): credential handling and the safety model.
+- [`docs/DEVICE_DATABASE.md`](docs/DEVICE_DATABASE.md): the device-capability
+  database.
+
+---
+
+## License
+
+This project is open source, MIT License, and available for personal and
+commercial use.
+
+---
+
+## Acknowledgments
+
+Built for the UniFi community, to help people run their networks instead of just
+photographing them.