queue-aiops 0.6.0__tar.gz → 0.8.0__tar.gz

{queue_aiops-0.6.0 → queue_aiops-0.8.0}/CHANGELOG.md

@@ -1,5 +1,19 @@
 # Changelog
 
+## v0.8.0 — 2026-08-10
+
+### Fixed
+- **An undetermined outcome no longer exits as a plain failure.** A write whose response was lost carries *both* `error` and `outcomeUnknown`, and the harness deliberately judges unknown first when writing the audit row — the change may have taken effect, so a blind retry could apply it twice. The CLI guard judged `error` first, so the audit said "may have taken effect" while the exit status told a script it had not happened. The two layers now agree (exit 2, not 1), and a test pins the ordering so it cannot silently flip back.
+- **The CLI reported a refused or failed governed write as a success.** 8 write call sites printed the governed twin's payload and exited **0** whatever it said — and `@tool_errors` flattens every refusal, guard rejection and upstream failure into `{"error": ...}` rather than raising, so nothing downstream of a `&&` chain or a CI step could tell a blocked write from a landed one. The dry-run path already exited non-zero, which made the asymmetry worse: the preview was stricter than the write it previews. Results now route through a `checked()` helper — exit 1 on an error payload, exit 2 on an undetermined outcome, unchanged on success. This defect class had been fixed repo-by-repo several times and kept coming back; an audit across the whole line found it live in **18 of the 24 tools at once (87 call sites)**, so each tool now carries an invariant test that fails if any future CLI command prints a governed result without checking it.
+
+## v0.7.0 — 2026-08-10
+
+### Fixed
+- **`purge_queue` could never have worked against a real RabbitMQ.** The management-API client sent `Accept: application/json`, but the purge endpoint (`DELETE /api/queues/{vhost}/{name}/contents`) answers *204 No Content* and offers no JSON representation, so content negotiation failed and the broker returned **406 with an empty body** — on every purge, on every server. Accept is now `application/json, */*`; every other endpoint (GETs, PUT declare, DELETE queue, DELETE policy) was enumerated against a live broker and behaves identically either way. Verified on RabbitMQ 3.13.7 by purging a real 9-message backlog.
+- **A 406 now explains itself.** The broker sends no body with a 406, so the generic branch produced a message with nothing in it — and, because the platform label already ends in "API", it read "RabbitMQ management API **API error**". There is now a 406 branch naming content negotiation as the cause, and the generic branch no longer doubles the word.
+- **Two more integer quantities stayed integers** (bug class #2): a node's `memUsedBytes` / `memLimitBytes` came back as floats (`161222656.0`) while the `diskFreeBytes` beside them was an int — one payload disagreeing with itself — and the per-peer `channels` count rendered as `1.0` for one channel, from a `0.0` accumulator seed. Both found on a live broker; regression tests assert the type.
+- **Write-path integer quantities stay integers** (bug class #2/#4). `ops/writes.py` still rendered whole-second and count fields through the float helper `num()` — `kill_client`'s `priorState.ageSeconds` came back as `49.0`, and the RabbitMQ purge/policy captures rendered `messages` and `priority` as floats too. It was missed in the earlier read-path sweep (the module did not even import `as_int`) and was inconsistent with `list_clients`, which already rendered the identical CLIENT LIST `age` field as an `int`. Now uses `as_int`; a regression test asserts the *type* (equality cannot catch `49 == 49.0`). Found live while verifying `kill-client` against a real blocked `BLPOP` client on Redis 7.4.
+
 ## v0.6.0 — 2026-08-03
 
 ### Fixed

{queue_aiops-0.6.0 → queue_aiops-0.8.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: queue-aiops
-Version: 0.6.0
+Version: 0.8.0
 Summary: Governed AI-ops for redis + rabbitmq: memory-pressure/latency/queue-backlog/connection-churn RCAs, slowlog and SCAN-budgeted big-key sampling, queues/policies/nodes, and governed writes (config set, client kill, purge/delete queue, policies) with a built-in governance harness (audit, budget, undo, risk tiers)
 Author-email: Wei Zhou <zhouwei008@gmail.com>
 License-Expression: MIT

queue_aiops-0.8.0/docs/VERIFICATION.md

@@ -0,0 +1,97 @@
+# Live verification status
+
+This document records what has and has not been validated against real Redis /
+RabbitMQ servers, so the maturity claim is auditable rather than a vibe.
+
+## Already live-verified ✅ — Redis 7.4.9, 2026-07-19
+
+Exercised end-to-end against a real **Redis 7.4.9** server (Docker) seeded with
+202 keys and a list-backed job queue:
+
+- `doctor` against a live server: PING OK, no-AUTH lab path reported clearly.
+- Every Redis read cross-checked against `redis-cli` ground truth:
+  `overview`, `redis info/memory/clients/slowlog/keyspace/bigkeys`
+  (key count matched `DBSIZE` exactly).
+- All four analyses ran clean: `analyze memory/latency/backlog/churn`.
+- Governance loop end-to-end: `redis_config_set` really changed
+  `maxmemory-policy`, captured `noeviction` as `priorState`, wrote an audit row,
+  and `undo_apply` restored the prior value on the live server.
+
+**A real defect was found and fixed by this run**: integer quantities were
+rendered as floats — `202.0` keys, `1.0` connected clients, float byte counts —
+because they were routed through `num()`. Values were arithmetically right but
+semantically wrong, and equality assertions could not catch it
+(`202 == 202.0`). Fixed with `as_int()` plus a regression test that asserts the
+*type*, keeping genuine ratios (`hitRatePct`, `usedPctOfMax`, `opsPerSec`) as
+floats. ⚠️ That round's claim of a fix "across the Redis and RabbitMQ reads"
+was **overstated**: the 2026-08-10 RabbitMQ run found `memUsedBytes` /
+`memLimitBytes` and the per-peer `channels` count still floats, and an earlier
+2026-08-03 round found the whole *write* path untouched. The sweep was never as
+wide as the sentence claimed — treat "fixed line-wide" claims as needing their
+own enumeration.
+
+## Not yet live-verified ⚠️
+
+- ~~**RabbitMQ** — the entire `rabbitmq` command group and its management-API
+  shapes are unit-tested only. This is now the largest gap in this repo.~~
+  **Closed 2026-08-10 against a real RabbitMQ 3.13.7** (management plugin), with
+  a seeded estate: 3 queues, a real backlog, a live pika consumer holding a
+  connection/channel, and a policy.
+  - All 7 reads cross-checked against `rabbitmqctl` / `rabbitmqadmin`:
+    `overview` (33 ready messages, cluster name, node count), `queues`
+    (per-queue counts and durability exact), `queue` detail, `connections`,
+    `channels` (unacked and consumer counts), `policies`, `nodes`.
+  - All 5 writes exercised end-to-end with their undo: `declare-queue` →
+    `undo_apply` deleted it; `set-policy` → `undo_apply` removed it leaving the
+    pre-existing policy untouched; `delete-queue` → `undo_apply` re-declared it
+    with the captured `durable=false`; `purge` destroyed a real 9-message
+    backlog and correctly recorded **no** undo.
+  - 🔴 **`purge_queue` had never worked against a real broker.** The HTTP client
+    sent `Accept: application/json`, and RabbitMQ's purge endpoint
+    (`DELETE /api/queues/{vhost}/{name}/contents`) answers *204 No Content* with
+    no JSON representation, so content negotiation failed and the broker
+    returned **406 with an empty body** — every purge, on every server. The
+    other endpoints were enumerated against the live broker and behave
+    identically under either Accept value, so the fix is `application/json, */*`.
+    The 406 message was also useless (no body to quote, and the label already
+    ended in "API" so it read "…management API API error"); both fixed.
+  - 🔴 Two integer quantities rendered as floats (bug class #2): a node's
+    `memUsedBytes` / `memLimitBytes` (`161222656.0` sitting next to an integer
+    `diskFreeBytes` — one payload disagreeing with itself), and the per-peer
+    `channels` count (`1.0` for one channel, from a `0.0` accumulator seed).
+  - Still untested: clustered RabbitMQ, quorum/stream queues, and TLS (AMQPS /
+    HTTPS management).
+- ~~Redis **cluster / sentinel** topologies (only standalone was exercised).~~
+  **Both exercised 2026-08-03**, and the cluster one found a defect: a node's
+  `totalKeys` was reported as the dataset total (100 on a 3-master cluster
+  holding 300). Reads, `overview` and the RCAs otherwise behaved correctly on a
+  cluster node, on a Sentinel-fronted primary, and on its replica (`role:
+  slave`). Still untested: an actual **failover** through Sentinel, and any
+  cluster-wide aggregate — this tool talks to one endpoint by design, which is
+  exactly why the scope marker matters.
+- ~~AUTH-enabled Redis and TLS connections.~~ **Both verified 2026-08-03**
+  against a real Redis 7.4 (Docker). AUTH: without the password every read
+  fails with a clear `Authentication required` envelope (nulls + `errors`, not
+  a silent empty); with the password via the encrypted-store/env fallback,
+  `overview`/`keyspace` matched `DBSIZE` exactly. TLS: `use_tls` with
+  `verify_ssl: false` connects over a `rediss://` socket to a self-signed
+  instance and returns the right key count; `verify_ssl: true` against the same
+  self-signed cert fails with `CERTIFICATE_VERIFY_FAILED` — TLS verification is
+  really enforced, not silently downgraded.
+- ~~`kill-client` against a real blocked client.~~ **Verified 2026-08-03**: a
+  client genuinely blocked on `BLPOP` was listed by `redis clients` (with the
+  tool's own connection correctly excluded), the `--dry-run` preview recorded a
+  `wouldKill` audit row *without* killing, the real kill removed it server-side
+  (confirmed via `CLIENT LIST`), and its CLIENT LIST row was captured as
+  `priorState`. Both the preview and the real call wrote audit rows through the
+  same governed twin.
+
+  **A real defect was found and fixed by this run** (bug class #2/#4): the write
+  path rendered integer quantities as floats — `ageSeconds: 49.0` in
+  `kill_client`'s `priorState`, and likewise `messages`/`priority` in the
+  RabbitMQ purge/policy captures — because `ops/writes.py` still routed them
+  through `num()`. It was missed in the earlier read-path sweep (it did not even
+  import `as_int`), and was inconsistent with `list_clients`, which already
+  rendered the identical CLIENT LIST `age` field as an `int`. Fixed with
+  `as_int()` and a type-asserting regression test (equality cannot catch it:
+  `49 == 49.0`); `ageSeconds` re-confirmed as `4` (int) on a live re-kill.

{queue_aiops-0.6.0 → queue_aiops-0.8.0}/pyproject.toml

@@ -4,7 +4,7 @@ build-backend = "hatchling.build"
 
 [project]
 name = "queue-aiops"
-version = "0.6.0"
+version = "0.8.0"
 description = "Governed AI-ops for redis + rabbitmq: memory-pressure/latency/queue-backlog/connection-churn RCAs, slowlog and SCAN-budgeted big-key sampling, queues/policies/nodes, and governed writes (config set, client kill, purge/delete queue, policies) with a built-in governance harness (audit, budget, undo, risk tiers)"
 readme = "README.md"
 license = "MIT"

{queue_aiops-0.6.0 → queue_aiops-0.8.0}/queue_aiops/cli/_common.py

@@ -76,6 +76,53 @@ def dry_run_print(*, operation: str, api_call: str, parameters: dict | None = No
     console.print("[magenta]  Run without --dry-run to execute.[/]\n")
 
 
+#: Exit status for a write whose outcome could not be determined. Kept distinct
+#: from 0 (confirmed) and 1 (failed) on purpose: a write whose response was lost
+#: is not a failure, but it is emphatically not a success either, and a script
+#: must be able to tell all three apart.
+EXIT_UNDETERMINED = 2
+
+
+def checked(result: Any) -> Any:
+    """Return ``result``, or abort when it reports a failed/undetermined write.
+
+    Every CLI command that calls a governed twin MUST pass the result through
+    here before printing its success line.
+
+    Governed twins are wrapped in ``@tool_errors``, which flattens any exception
+    into ``{"error": ...}`` and **returns** it. The CLI therefore never sees the
+    exception, so a command that prints its result unconditionally reports a
+    refused or failed operation exactly like a successful one — and exits 0, so
+    a script cannot tell either. The dry-run path already refused with a
+    non-zero status, which made the asymmetry worse: the preview was stricter
+    than the real call.
+
+    ``outcomeUnknown`` (set by the harness when a write's response is lost) is
+    neither success nor failure — the change may still have landed. It gets its
+    own line and :data:`EXIT_UNDETERMINED`, never a silent success.
+    """
+    if not isinstance(result, dict):
+        return result
+    error = result.get("error")
+    # ``outcomeUnknown`` is judged BEFORE ``error``, matching the harness: a
+    # write whose response was lost carries BOTH keys, and it is audited
+    # `unknown` precisely because it may have taken effect. Reporting that as a
+    # plain failure would tell a script the change did not happen and invite the
+    # double-apply the payload's own note warns about.
+    if result.get("outcomeUnknown"):
+        console.print(
+            f"[yellow]Outcome undetermined: {result.get('note') or ''}[/]"
+        )
+        raise typer.Exit(EXIT_UNDETERMINED)
+    if error:
+        console.print(f"[red]Error: {error}[/]")
+        hint = result.get("hint")
+        if hint:
+            console.print(f"[dim]{hint}[/]")
+        raise typer.Exit(1)
+    return result
+
+
 def dry_run_preview(
     preview: Any, *, operation: str, api_call: str, parameters: dict | None = None
 ) -> None:

{queue_aiops-0.6.0 → queue_aiops-0.8.0}/queue_aiops/cli/rabbit_cmds.py

@@ -10,6 +10,7 @@ import typer
 from queue_aiops.cli._common import (
     DryRunOption,
     TargetOption,
+    checked,
     cli_errors,
     console,
     double_confirm,
@@ -157,7 +158,7 @@ def rabbit_purge(
             parameters={"vhost": vhost, "name": name})
         return
     double_confirm("purge queue", name)
-    console.print_json(json.dumps(gov.purge_queue(vhost=vhost, name=name, target=target)))
+    console.print_json(json.dumps(checked(gov.purge_queue(vhost=vhost, name=name, target=target))))
 
 
 @rabbitmq_app.command("delete-queue")
@@ -180,7 +181,7 @@ def rabbit_delete_queue(
             parameters={"vhost": vhost, "name": name})
         return
     double_confirm("delete queue", name)
-    console.print_json(json.dumps(gov.delete_queue(vhost=vhost, name=name, target=target)))
+    console.print_json(json.dumps(checked(gov.delete_queue(vhost=vhost, name=name, target=target))))
 
 
 @rabbitmq_app.command("declare-queue")
@@ -211,9 +212,9 @@ def rabbit_declare_queue(
                         "auto_delete": auto_delete})
         return
     double_confirm("declare queue", name)
-    console.print_json(json.dumps(gov.declare_queue(
+    console.print_json(json.dumps(checked(gov.declare_queue(
         vhost=vhost, name=name, durable=durable, auto_delete=auto_delete, target=target
-    )))
+    ))))
 
 
 @rabbitmq_app.command("set-policy")
@@ -252,10 +253,10 @@ def rabbit_set_policy(
                         "apply_to": apply_to})
         return
     double_confirm("set policy", name)
-    console.print_json(json.dumps(gov.set_policy(
+    console.print_json(json.dumps(checked(gov.set_policy(
         vhost=vhost, name=name, pattern=pattern, definition=definition_obj,
         priority=priority, apply_to=apply_to, target=target,
-    )))
+    ))))
 
 
 @rabbitmq_app.command("delete-policy")
@@ -278,4 +279,6 @@ def rabbit_delete_policy(
             parameters={"vhost": vhost, "name": name})
         return
     double_confirm("delete policy", name)
-    console.print_json(json.dumps(gov.delete_policy(vhost=vhost, name=name, target=target)))
+    console.print_json(
+        json.dumps(checked(gov.delete_policy(vhost=vhost, name=name, target=target)))
+    )

{queue_aiops-0.6.0 → queue_aiops-0.8.0}/queue_aiops/cli/redis_cmds.py

@@ -10,6 +10,7 @@ import typer
 from queue_aiops.cli._common import (
     DryRunOption,
     TargetOption,
+    checked,
     cli_errors,
     console,
     double_confirm,
@@ -138,7 +139,7 @@ def redis_config_set(
         return
     double_confirm("set config parameter", parameter)
     console.print_json(
-        json.dumps(gov.redis_config_set(parameter=parameter, value=value, target=target))
+        json.dumps(checked(gov.redis_config_set(parameter=parameter, value=value, target=target)))
     )
 
 
@@ -165,5 +166,5 @@ def redis_kill_client(
         return
     double_confirm("kill client", who or "(unspecified)")
     console.print_json(
-        json.dumps(gov.redis_kill_client(client_id=client_id, addr=addr, target=target))
+        json.dumps(checked(gov.redis_kill_client(client_id=client_id, addr=addr, target=target)))
     )

{queue_aiops-0.6.0 → queue_aiops-0.8.0}/queue_aiops/cli/undo.py

@@ -15,6 +15,7 @@ import typer
 from queue_aiops.cli._common import (
     DryRunOption,
     TargetOption,
+    checked,
     cli_errors,
     console,
     double_confirm,
@@ -65,4 +66,4 @@ def undo_apply_cmd(
         )
         return
     double_confirm("apply undo", undo_id)
-    console.print_json(json.dumps(gov.undo_apply(undo_id=undo_id, target=target)))
+    console.print_json(json.dumps(checked(gov.undo_apply(undo_id=undo_id, target=target))))

{queue_aiops-0.6.0 → queue_aiops-0.8.0}/queue_aiops/connection.py

@@ -63,12 +63,25 @@ def _teaching_message(status: int, path: str, body: str, label: str) -> str:
             f"Bad request (400) on {label} {path}. The broker rejected the "
             f"request — check required fields and value formats. {snippet}"
         )
+    if status == 406:
+        # Content negotiation, and the broker sends NO body with it — so the
+        # generic branch produced a message that explained nothing at all
+        # (measured while purge_queue was failing on RabbitMQ 3.13.7).
+        return (
+            f"Content negotiation failed (406) on {label} {path}. The endpoint "
+            f"has no representation matching the request's Accept header — "
+            f"endpoints that answer '204 No Content' need '*/*' in Accept, not "
+            f"'application/json' alone. The broker sends no body with a 406, so "
+            f"there is nothing further to quote. {snippet}"
+        ).strip()
     if status in (500, 502, 503, 504):
         return (
             f"{label} server error ({status}) on {path}. The broker may be "
             f"busy; retry shortly. {snippet}"
         )
-    return f"{label} API error ({status}) on {path}. {snippet}"
+    # `label` already ends in "API" for rabbitmq ("RabbitMQ management API"),
+    # so the old "{label} API error" read "…management API API error".
+    return f"{label} error ({status}) on {path}. {snippet}".strip()
 
 
 class QueueConnection:
@@ -102,7 +115,19 @@ class QueueConnection:
             verify=target.verify_ssl,
             timeout=_TIMEOUT,
             auth=httpx.BasicAuth(target.username, target.secret),
-            headers={"Accept": "application/json", "Content-Type": "application/json"},
+            # `*/*` must stay in Accept. RabbitMQ's purge endpoint
+            # (DELETE /api/queues/{vhost}/{name}/contents) answers 204 with no
+            # body and offers no JSON representation, so a bare
+            # "Accept: application/json" fails content negotiation and the
+            # server returns **406 with an empty body** — measured on 3.13.7.
+            # purge_queue could therefore never succeed against a real broker.
+            # Every other endpoint (GETs, PUT declare, DELETE queue, DELETE
+            # policy) behaves identically under either value; enumerated, not
+            # assumed.
+            headers={
+                "Accept": "application/json, */*",
+                "Content-Type": "application/json",
+            },
         )
 
     @property

{queue_aiops-0.6.0 → queue_aiops-0.8.0}/queue_aiops/ops/rabbit_reads.py

@@ -132,7 +132,9 @@ def list_connections(conn: Any) -> dict:
         by_host: dict[str, dict] = {}
         for r in rows:
             bucket = by_host.setdefault(
-                r["peerHost"], {"peerHost": r["peerHost"], "connections": 0, "channels": 0.0}
+                # A channel count seeded as 0.0 makes the whole sum a float —
+                # "1.0 channels" for one channel. Both members are counts.
+                r["peerHost"], {"peerHost": r["peerHost"], "connections": 0, "channels": 0}
             )
             bucket["connections"] += 1
             bucket["channels"] += r["channels"]
@@ -211,8 +213,12 @@ def node_health(conn: Any) -> dict:
     try:
         rows = []
         for n in conn.platform.rows(conn.get(conn.platform.path("nodes"))):
-            mem_used = num(n.get("mem_used"))
-            mem_limit = num(n.get("mem_limit"))
+            # Byte counts, not ratios — every sibling byte field below already
+            # used as_int, so routing these two through the float helper made a
+            # single payload disagree with itself (memUsedBytes 161222656.0
+            # next to diskFreeBytes 36616110080).
+            mem_used = as_int(n.get("mem_used"))
+            mem_limit = as_int(n.get("mem_limit"))
             rows.append(
                 {
                     "name": opt(n.get("name"), 64),

{queue_aiops-0.6.0 → queue_aiops-0.8.0}/queue_aiops/ops/writes.py

@@ -42,7 +42,7 @@ import re
 from typing import Any
 
 from queue_aiops.connection import QueueApiError
-from queue_aiops.ops._util import as_obj, num, opt, s
+from queue_aiops.ops._util import as_int, as_obj, opt, s
 
 # CONFIG SET parameter names are plain words (e.g. maxmemory-policy); reject
 # anything else before it reaches the wire.
@@ -220,7 +220,7 @@ def kill_client(conn: Any, client_id: int = 0, addr: str = "") -> dict:
                 "addr": opt(c.get("addr"), 64),
                 "name": opt(c.get("name"), 64),
                 "lastCommand": opt(c.get("cmd"), 64),
-                "ageSeconds": num(c.get("age")),
+                "ageSeconds": as_int(c.get("age")),
             }
             break
     killed = (
@@ -246,7 +246,7 @@ def _queue_def(conn: Any, vhost: str, name: str) -> dict:
         conn.platform.normalise(conn.get(conn.platform.path("queue", vhost=vhost, name=name)))
     )
     return {
-        "messages": num(obj.get("messages")),
+        "messages": as_int(obj.get("messages")),
         "durable": bool(obj.get("durable")),
         "autoDelete": bool(obj.get("auto_delete")),
         "arguments": as_obj(obj.get("arguments")),
@@ -350,7 +350,7 @@ def _policy_or_none(conn: Any, vhost: str, name: str) -> dict | None:
     return {
         "pattern": opt(obj.get("pattern"), 128),
         "definition": as_obj(obj.get("definition")),
-        "priority": num(obj.get("priority")),
+        "priority": as_int(obj.get("priority")),
         "applyTo": opt(obj.get("apply-to") or obj.get("apply_to") or "all", 32),
     }
 

{queue_aiops-0.6.0 → queue_aiops-0.8.0}/server.json

@@ -7,12 +7,12 @@
     "url": "https://github.com/AIops-tools/Queue-AIops",
     "source": "github"
   },
-  "version": "0.6.0",
+  "version": "0.8.0",
   "packages": [
     {
       "registryType": "pypi",
       "identifier": "queue-aiops",
-      "version": "0.6.0",
+      "version": "0.8.0",
       "transport": {
         "type": "stdio"
       }

queue_aiops-0.8.0/tests/test_cli_governed_exit.py

@@ -0,0 +1,167 @@
+"""The CLI must never report a failed governed write as a success.
+
+Two regressions are pinned here.
+
+**The behaviour** — ``checked()`` aborts non-zero on an error payload and on an
+undetermined outcome. Governed twins are wrapped in ``@tool_errors``, which
+flattens every exception into ``{"error": ...}`` and *returns* it, so a
+command that prints its result unconditionally exits 0 for an operation that
+was refused or failed.
+
+**The invariant** — every CLI call into a governed write twin routes its result
+through that helper. This is the part that matters over time: the defect was
+fixed repo-by-repo several times and kept coming back, because nothing stopped
+a *new* command from printing a governed result bare. A structural assertion
+does. It was introduced after an audit found the preview path guarded and the
+real write path unguarded in 18 of the 24 tools at once — the preview was
+stricter than the write it previews.
+"""
+
+from __future__ import annotations
+
+import ast
+import pathlib
+
+import pytest
+import typer
+
+from queue_aiops.cli._common import checked
+
+CLI_DIR = pathlib.Path(__file__).resolve().parents[1] / "queue_aiops" / "cli"
+TOOLS_DIR = pathlib.Path(__file__).resolve().parents[1] / "mcp_server" / "tools"
+
+#: Helpers that end the command non-zero when the payload reports a failure.
+CHECKERS = ['checked', 'dry_run_preview']
+
+
+@pytest.mark.unit
+def test_checked_aborts_on_error_payload():
+    with pytest.raises(typer.Exit) as exc:
+        checked({"error": "refused: that would lock this tool out"})
+    assert exc.value.exit_code == 1
+
+
+@pytest.mark.unit
+def test_checked_passes_a_successful_payload_through():
+    payload = {"action": "did_a_thing", "ok": True}
+    assert checked(payload) == payload
+
+
+def _callee(node: ast.AST) -> str | None:
+    if isinstance(node, ast.Name):
+        return node.id
+    if isinstance(node, ast.Attribute):
+        return node.attr
+    return None
+
+
+def _write_tool_names() -> set[str]:
+    names: set[str] = set()
+    for path in TOOLS_DIR.glob("*.py"):
+        tree = ast.parse(path.read_text())
+        for node in ast.walk(tree):
+            if isinstance(node, ast.FunctionDef) and "[WRITE]" in (ast.get_docstring(node) or ""):
+                names.add(node.name)
+    return names
+
+
+def _aborts_on_failure(fn: ast.FunctionDef) -> bool:
+    """Does this function raise ``typer.Exit`` off a failure key in the payload?
+
+    Several commands guard with a locally named helper (``_require_ok``,
+    ``_emit``, ...) rather than the shared one. Those are correct guards, so the
+    invariant recognises the *shape* instead of a list of blessed names.
+    """
+    raises_exit = any(
+        isinstance(n, ast.Call) and _callee(n.func) == "Exit" for n in ast.walk(fn)
+    )
+    reads_failure_key = any(
+        isinstance(n, ast.Constant) and n.value in ("error", "outcomeUnknown")
+        for n in ast.walk(fn)
+    )
+    return raises_exit and reads_failure_key
+
+
+def _checker_names() -> set[str]:
+    """CHECKERS, every local guard, and any wrapper delegating to one of them."""
+    funcs: dict[str, list[ast.FunctionDef]] = {}
+    for path in CLI_DIR.rglob("*.py"):
+        for node in ast.walk(ast.parse(path.read_text())):
+            if isinstance(node, ast.FunctionDef):
+                funcs.setdefault(node.name, []).append(node)
+    found = set(CHECKERS) | {
+        name for name, defs in funcs.items() if any(_aborts_on_failure(f) for f in defs)
+    }
+    changed = True
+    while changed:
+        changed = False
+        for name, defs in funcs.items():
+            if name in found:
+                continue
+            for fn in defs:
+                if any(isinstance(n, ast.Call) and _callee(n.func) in found for n in ast.walk(fn)):
+                    found.add(name)
+                    changed = True
+                    break
+    return found
+
+
+@pytest.mark.unit
+def test_every_cli_governed_write_result_is_checked():
+    tools = _write_tool_names()
+    assert tools, "no [WRITE] tools parsed — the invariant would vacuously pass"
+    checkers = _checker_names()
+    unchecked: list[str] = []
+
+    for path in sorted(CLI_DIR.rglob("*.py")):
+        tree = ast.parse(path.read_text())
+        for func in [n for n in ast.walk(tree) if isinstance(n, ast.FunctionDef)]:
+            parents: dict[ast.AST, ast.AST] = {}
+            for node in ast.walk(func):
+                for child in ast.iter_child_nodes(node):
+                    parents[child] = node
+            guarded = {
+                arg.id
+                for node in ast.walk(func)
+                if isinstance(node, ast.Call) and _callee(node.func) in checkers
+                for arg in list(node.args) + [kw.value for kw in node.keywords]
+                if isinstance(arg, ast.Name)
+            }
+            for node in ast.walk(func):
+                if not (isinstance(node, ast.Call) and _callee(node.func) in tools):
+                    continue
+                parent = parents.get(node)
+                if isinstance(parent, (ast.keyword, ast.Starred)):
+                    parent = parents.get(parent)
+                if isinstance(parent, ast.Call) and _callee(parent.func) in checkers:
+                    continue
+                if isinstance(parent, ast.Assign) and any(
+                    isinstance(t, ast.Name) and t.id in guarded for t in parent.targets
+                ):
+                    continue
+                unchecked.append(f"{path.name}:{node.lineno}:{_callee(node.func)}")
+
+    assert not unchecked, (
+        "these CLI call sites print a governed write's result without routing it "
+        f"through {sorted(checkers)}, so a refusal exits 0: {unchecked}"
+    )
+
+@pytest.mark.unit
+def test_undetermined_outcome_beats_error_and_is_not_a_plain_failure():
+    """A lost response carries BOTH keys — it must exit 2, not 1.
+
+    ``@tool_errors`` flattens the exception into ``{"error": ...}`` and, when
+    the failure is one where the write may still have landed, also sets
+    ``outcomeUnknown``. The harness deliberately judges unknown *before* error
+    when writing the audit row, for the reason its own note gives: the change
+    may have taken effect, so a blind retry could apply it twice. The CLI has to
+    agree, or the audit says "may have taken effect" while the exit status tells
+    a script it did not happen.
+    """
+    with pytest.raises(typer.Exit) as exc:
+        checked({
+            "error": "connection reset while awaiting the response",
+            "outcomeUnknown": True,
+            "note": "may have taken effect",
+        })
+    assert exc.value.exit_code == 2

{queue_aiops-0.6.0 → queue_aiops-0.8.0}/tests/test_connection.py

@@ -239,3 +239,56 @@ def test_manager_unknown_target_teaches_available(fake_build):
     mgr = ConnectionManager(_cfg())
     with pytest.raises(KeyError, match="Available: cache1, broker1"):
         mgr.connect("ghost")
+
+
+@pytest.mark.unit
+def test_rabbitmq_accept_header_keeps_a_wildcard_for_no_content_endpoints(monkeypatch):
+    """Accept must not be JSON-only, or the purge endpoint 406s.
+
+    Measured on a real RabbitMQ 3.13.7: ``DELETE /api/queues/{vhost}/{name}
+    /contents`` answers 204 with no body and offers no JSON representation, so
+    a bare ``Accept: application/json`` fails content negotiation and the broker
+    returns **406 with an empty body** — meaning ``purge_queue`` could never
+    succeed against a real server. Adding ``*/*`` returns 204, and every other
+    endpoint (GETs, PUT declare, DELETE queue, DELETE policy) answers the same
+    either way; that was enumerated against the live broker, not assumed.
+
+    Nothing else pins this header, which is why the defect survived: the fake
+    HTTP client in the unit suite never negotiates content.
+    """
+    monkeypatch.setenv("QUEUE_BROKER1_SECRET", "pw")
+    client = QueueConnection._build_client(rabbit_target())
+    try:
+        accept = client.headers["accept"]
+    finally:
+        client.close()
+    assert "application/json" in accept
+    assert "*/*" in accept, (
+        "purge (DELETE .../contents) 406s when Accept is JSON-only — keep */*"
+    )
+
+
+@pytest.mark.unit
+def test_rabbitmq_406_explains_content_negotiation():
+    """A 406 carries no body, so the generic branch explained nothing.
+
+    Encountered live while purge_queue was failing: the operator got
+    "RabbitMQ management API API error (406) on /api/queues/%2F/q/contents. "
+    — a doubled word and zero diagnosis. The 406 branch now names the cause,
+    and the generic branch no longer repeats "API" after a label ending in it.
+    """
+    conn = rabbit_conn({("GET", "/api/x"): FakeResponse(406, text="")})
+    with pytest.raises(QueueApiError) as err:
+        conn.get("/api/x")
+    msg = str(err.value)
+    assert "Content negotiation failed (406)" in msg
+    assert "Accept" in msg
+    assert "API API error" not in msg
+
+
+@pytest.mark.unit
+def test_rabbitmq_unmapped_status_does_not_double_the_word_api():
+    conn = rabbit_conn({("GET", "/api/x"): FakeResponse(418, text="teapot")})
+    with pytest.raises(QueueApiError) as err:
+        conn.get("/api/x")
+    assert "API API error" not in str(err.value)

{queue_aiops-0.6.0 → queue_aiops-0.8.0}/tests/test_reads.py

@@ -239,6 +239,9 @@ def test_list_connections_groups_by_peer_host():
     assert out["total"] == 3
     assert out["byPeerHost"][0]["peerHost"] == "10.0.0.7"
     assert out["byPeerHost"][0]["channels"] == 4
+    # A channel count, not a rate: seeding the accumulator at 0.0 rendered one
+    # channel as "1.0" on a live broker. Equality cannot catch it (4 == 4.0).
+    assert isinstance(out["byPeerHost"][0]["channels"], int)
 
 
 @pytest.mark.unit
@@ -274,6 +277,12 @@ def test_node_health_flags_alarms():
     out = rabbit_reads.node_health(conn)
     assert out["alarms"] == 1
     assert out["nodes"][0]["memAlarm"] is True
+    # Byte counts stay integers. On a live broker these two came back as
+    # 161222656.0 / 13462616473.0 while diskFreeBytes beside them was an int —
+    # one payload disagreeing with itself (bug class #2).
+    node = out["nodes"][0]
+    for field in ("memUsedBytes", "memLimitBytes", "diskFreeBytes"):
+        assert isinstance(node[field], int), field
 
 
 @pytest.mark.unit

{queue_aiops-0.6.0 → queue_aiops-0.8.0}/tests/test_writes.py

@@ -71,6 +71,42 @@ def test_kill_client_requires_id_or_addr():
         ops.kill_client(conn)
 
 
+@pytest.mark.unit
+def test_write_prior_state_integer_quantities_stay_int():
+    """priorState counts/whole-seconds must be int, not float (bug class #2/#4).
+
+    CLIENT LIST ``age`` (whole seconds), a queue's ``messages`` count and a
+    policy ``priority`` are integer quantities. The write path rendered them
+    through ``num`` → ``120.0``/``1234.0``/``3.0`` — inconsistent with the read
+    path (``list_clients`` already uses ``as_int``) and semantically wrong. An
+    equality assertion cannot catch this (``120 == 120.0``); assert the *type*.
+    ``bool`` must not sneak through the int check (it subclasses int).
+    """
+    kc = ops.kill_client(
+        redis_conn(clients=[{"id": "77", "addr": "10.0.0.5:5000", "cmd": "blpop",
+                             "age": "120"}]),
+        client_id=77,
+    )
+    age = kc["priorState"]["client"]["ageSeconds"]
+    assert isinstance(age, int) and not isinstance(age, bool) and age == 120
+
+    purge = ops.purge_queue(
+        rabbit_conn({("GET", "/api/queues/%2F/orders"): _Q,
+                     ("DELETE", "/api/queues/%2F/orders/contents"): {}}),
+        "/", "orders",
+    )
+    msgs = purge["priorState"]["messages"]
+    assert isinstance(msgs, int) and not isinstance(msgs, bool) and msgs == 1234
+
+    policy = ops.set_policy(
+        rabbit_conn({("GET", "/api/policies/%2F/lim"): _POLICY,
+                     ("PUT", "/api/policies/%2F/lim"): {}}),
+        "/", "lim", "^work\\.", {"max-length": 9000}, priority=4,
+    )
+    prio = policy["priorState"]["policy"]["priority"]
+    assert isinstance(prio, int) and not isinstance(prio, bool) and prio == 3
+
+
 # ── rabbitmq purge/delete prior-state capture ────────────────────────────────
 
 _Q = {"name": "orders", "vhost": "/", "messages": 1234, "durable": True,

{queue_aiops-0.6.0 → queue_aiops-0.8.0}/uv.lock

@@ -812,7 +812,7 @@ wheels = [
 
 [[package]]
 name = "queue-aiops"
-version = "0.6.0"
+version = "0.8.0"
 source = { editable = "." }
 dependencies = [
     { name = "cryptography" },

queue_aiops-0.6.0/docs/VERIFICATION.md

@@ -1,41 +0,0 @@
-# Live verification status
-
-This document records what has and has not been validated against real Redis /
-RabbitMQ servers, so the maturity claim is auditable rather than a vibe.
-
-## Already live-verified ✅ — Redis 7.4.9, 2026-07-19
-
-Exercised end-to-end against a real **Redis 7.4.9** server (Docker) seeded with
-202 keys and a list-backed job queue:
-
-- `doctor` against a live server: PING OK, no-AUTH lab path reported clearly.
-- Every Redis read cross-checked against `redis-cli` ground truth:
-  `overview`, `redis info/memory/clients/slowlog/keyspace/bigkeys`
-  (key count matched `DBSIZE` exactly).
-- All four analyses ran clean: `analyze memory/latency/backlog/churn`.
-- Governance loop end-to-end: `redis_config_set` really changed
-  `maxmemory-policy`, captured `noeviction` as `priorState`, wrote an audit row,
-  and `undo_apply` restored the prior value on the live server.
-
-**A real defect was found and fixed by this run**: integer quantities were
-rendered as floats — `202.0` keys, `1.0` connected clients, float byte counts —
-because they were routed through `num()`. Values were arithmetically right but
-semantically wrong, and equality assertions could not catch it
-(`202 == 202.0`). Fixed with `as_int()` across the Redis and RabbitMQ reads,
-plus a regression test that asserts the *type*, keeping genuine ratios
-(`hitRatePct`, `usedPctOfMax`, `opsPerSec`) as floats.
-
-## Not yet live-verified ⚠️
-
-- **RabbitMQ** — the entire `rabbitmq` command group and its management-API
-  shapes are unit-tested only. This is now the largest gap in this repo.
-- ~~Redis **cluster / sentinel** topologies (only standalone was exercised).~~
-  **Both exercised 2026-08-03**, and the cluster one found a defect: a node's
-  `totalKeys` was reported as the dataset total (100 on a 3-master cluster
-  holding 300). Reads, `overview` and the RCAs otherwise behaved correctly on a
-  cluster node, on a Sentinel-fronted primary, and on its replica (`role:
-  slave`). Still untested: an actual **failover** through Sentinel, and any
-  cluster-wide aggregate — this tool talks to one endpoint by design, which is
-  exactly why the scope marker matters.
-- AUTH-enabled Redis and TLS connections.
-- `kill-client` against a real blocked client.