mqgov-cli 0.1.0 → 0.3.0

package/README.md

@@ -4,7 +4,7 @@
 
 **Governed message-broker operations for humans _and_ AI agents.**
 
-One safe command line for **Kafka**, **RabbitMQ**, **Pulsar**, and **RocketMQ** — list, describe, peek, produce, reset offsets, purge, and delete topics without ever fat-fingering production or silently draining a queue.
+One safe command line for **Kafka**, **RabbitMQ**, **Pulsar**, and **RocketMQ** — list, describe, peek, tail, produce, govern DLQs, reset offsets, inspect ACLs and schemas, purge, and delete topics without ever fat-fingering production or silently draining a queue.
 
 [![npm version](https://img.shields.io/npm/v/mqgov-cli.svg)](https://www.npmjs.com/package/mqgov-cli)
 [![CI](https://github.com/JiangHe12/mqgov-cli/actions/workflows/ci.yml/badge.svg)](https://github.com/JiangHe12/mqgov-cli/actions/workflows/ci.yml)
@@ -25,7 +25,7 @@ Message brokers — **Kafka**, **RabbitMQ**, **Pulsar**, **RocketMQ** — are th
 
 - 🔎 **Shows you the blast radius first** — `--dry-run` / `--plan` print the exact per-partition impact (how many messages an offset reset will replay or skip) before anything happens.
 - 🛡️ **Refuses to do something dangerous without explicit sign-off** — risky commands need a confirmation flag, a change ticket, and an explicit `--allow-*` for the operation.
-- 👀 **Peeks without consuming** — inspecting messages never advances a consumer's position or drains a queue.
+- 👀 **Peeks/tails without consuming** — inspecting or streaming message fingerprints never advances a consumer's position or drains a queue.
 - 📜 **Records everything in a tamper-evident audit log** — sha256 fingerprints and counts only, **never your message bodies**.
 - 🤖 **Is safe to hand to an AI agent** — the agent can read and preview freely, but **cannot** invent the human approvals required for dangerous actions.
 
@@ -38,10 +38,10 @@ It's built on the shared [`opskit-core`](https://github.com/JiangHe12/opskit-cor
 | | |
 |---|---|
 | 📨 **Four brokers** | **Kafka** (franz-go), **RabbitMQ** (AMQP + management API), **Pulsar** (client + admin REST), **RocketMQ** (rocketmq-client-go/v2). One backend-agnostic governance model; pick per context or override per command. |
-| 🧱 **topic / group / message** | topics: list · describe · create · alter · delete · purge. consumer groups: list · lag · reset-offset. messages: non-destructive peek · produce. |
+| 🧱 **topic / group / message / dlq / acl / schema / fleet** | topics: list · describe · create · alter · delete · purge. consumer groups: list · lag · reset-offset. messages: non-destructive peek · tail · produce. DLQs: list · peek · redrive · purge through native broker models. ACLs: list · grant · revoke where supported. Schemas: list · describe · check where native schema registry support exists. Fleet: read-only status and topic inventory across configured contexts. |
 | 🔐 **R0–R3 governance** | every operation is risk-classified by the fail-closed `mqclass` engine; protected contexts and internal/system topics escalate one tier; AI callers can never self-authorize. |
 | 🎯 **Real blast-radius preview** | `reset-offset --dry-run` and `purge --dry-run` compute the actual per-partition message delta from the live broker — no guessing. The preview is read-only and never mutates. |
-| 👀 **Non-destructive peek** | inspect messages as fingerprints without consuming them or moving any cursor (Pulsar Reader, RabbitMQ get+requeue). Where a broker can't guarantee this, peek fails closed rather than silently consuming. |
+| 👀 **Non-destructive peek/tail** | inspect or stream messages as fingerprints without consuming them or moving any cursor (Kafka direct partition reads, Pulsar Reader, RabbitMQ get+requeue for peek only). Where a broker can't guarantee this, the operation fails closed rather than silently consuming. |
 | 🧭 **Honest capabilities** | brokers differ — mqgov reports what each one actually supports (`capabilities -o json`) and **fails closed with `NOT_IMPLEMENTED`** for the rest, never faking it. |
 | 📜 **Tamper-evident audit** | hash-chained log of every action (sha256 fingerprints + counts, **no message bodies/keys/headers**); `audit verify` detects tampering. |
 | 🩺 **Ops & DX** | backend-bound `ctx` contexts with credstore-backed secrets, `doctor` diagnostics, shell `completion`, OpenTelemetry traces/metrics, JSON output everywhere. |
@@ -54,11 +54,17 @@ It's built on the shared [`opskit-core`](https://github.com/JiangHe12/opskit-cor
 | topic list / describe / create / delete | ✅ | ✅ | ✅ | ✅ |
 | produce | ✅ | ✅ | ✅ | ✅ |
 | **non-destructive peek** | ✅ | ✅ (Reader) | ✅ (get+requeue) | ❌ `NOT_IMPLEMENTED`¹ |
+| **non-destructive tail** | ✅ | ✅ (Reader) | ❌ `NOT_IMPLEMENTED`² | ❌ `NOT_IMPLEMENTED`¹ |
 | **offset lag / reset** | ✅ | ✅ (cursor) | ❌ (no offsets) | ❌ |
 | alter partitions | ✅ | ✅ | ❌ | ❌ |
 | purge | ✅ | ✅ | ✅ | ❌ |
+| **DLQ list / peek / redrive / purge** | list ❌; explicit topic peek/redrive/purge ✅ | ✅ `{topic}-{subscription}-DLQ` | ✅ DLX queues | list ✅ `%DLQ%group`; others ❌ |
+| **ACL list / grant / revoke** | ✅ | ✅ namespace/topic permissions | ✅ user-vhost permissions | ❌ `NOT_IMPLEMENTED`³ |
+| **schema list / describe / check** | ✅ Confluent Schema Registry | ✅ built-in admin schema API | ❌ `NOT_IMPLEMENTED` | ❌ `NOT_IMPLEMENTED` |
 
-¹ RocketMQ's Go v2 `PullConsumer` enters the consumer-group lifecycle and commits offsets, so it cannot guarantee a non-destructive peek — mqgov fails closed instead of silently advancing offsets. Unsupported operations always return `NOT_IMPLEMENTED` (exit 12), never a fake success.
+¹ RocketMQ's Go v2 `PullConsumer` enters the consumer-group lifecycle and commits offsets, so it cannot guarantee non-destructive peek/tail — mqgov fails closed instead of silently advancing offsets. ² RabbitMQ has no forward non-destructive tail because reads are consume/requeue oriented. Unsupported operations always return `NOT_IMPLEMENTED` (exit 12), never a fake success.
+
+³ RocketMQ broker ACLs live in broker-side `plain_acl.yml`, but `rocketmq-client-go/v2` does not expose a public, cgo-free admin API for reading or changing that config. mqgov does not shell out to the Java `mqadmin` tool and does not hand-roll remoting commands; manage RocketMQ ACLs out of band with broker configuration or official mqadmin until the Go client exposes a clean API.
 
 ---
 
@@ -100,6 +106,7 @@ mqgov ctx test                       # ping the broker through the context
 mqgov topic list -o json
 mqgov topic describe orders -o json
 mqgov message peek orders --count 5 -o json     # fingerprints only, nothing consumed
+mqgov message tail orders --max-messages 10 -o json
 
 # 3. Preview the blast radius of a dangerous op — nothing is changed yet
 mqgov group reset-offset billing orders --to latest --dry-run -o json   # shows per-partition delta
@@ -121,19 +128,19 @@ Every command is sorted into one of four **risk tiers** by the fail-closed `mqcl
 
 | Tier | What it covers | What you must provide |
 |:---:|---|---|
-| **R0** | Reads & previews (`topic list/describe`, `group list/lag`, `message peek`, `*-dry-run`, `audit query/verify`, `doctor`) | Nothing — but it's still audited |
+| **R0** | Reads & previews (`topic list/describe`, `group list/lag`, `message peek`, `message tail`, `dlq list/peek`, `acl list`, `schema list/describe/check`, `fleet status/topics`, `*-dry-run`, `audit query/verify`, `doctor`) | Nothing — but it's still audited |
 | **R1** | Ordinary writes (`message produce`, `topic create`) | `--yes` (or an interactive confirmation) |
-| **R2** | Elevated mutations (`topic alter`, `group create/delete`, produce to a **protected** topic) | `--yes` **and** a non-empty `--ticket` |
-| **R3** | Destructive / irreversible (`group reset-offset`, `topic purge`, `topic delete`, produce to an **internal/system** topic) | The above **plus** the exact `--allow-*` flag |
+| **R2** | Elevated mutations (`topic alter`, `group create/delete`, `acl grant`, produce to a **protected** topic) | `--yes` **and** a non-empty `--ticket` |
+| **R3** | Destructive / irreversible (`group reset-offset`, `topic purge`, `topic delete`, `dlq redrive`, `dlq purge`, broad `acl grant`, `acl revoke`, produce to an **internal/system** topic) | The above **plus** the exact `--allow-*` flag |
 
-The R3 allow flags: `--allow-offset-reset`, `--allow-topic-purge`, `--allow-topic-delete`, `--allow-internal-produce`.
+The R3 allow flags: `--allow-offset-reset`, `--allow-topic-purge`, `--allow-topic-delete`, `--allow-destructive-acl`, `--allow-internal-produce`.
 
 **Protected contexts, protected topics, and internal/system topics raise the tier by one.** For example, producing to `__consumer_offsets` is treated as a destructive R3 operation and needs `--allow-internal-produce`.
 
 Three rules keep this safe — especially for automation:
 
 1. **Blast radius comes from the tool, not a guess.** Use `--dry-run` / `--plan` to see the exact per-partition impact. Never estimate it by reasoning.
-2. **`mqclass` is fail-closed and structure-aware.** All offset changes, purge, and delete are pinned R3; wildcard/glob targets escalate; an unknown operation fails closed to the highest tier — it never falls to R0.
+2. **`mqclass` is fail-closed and structure-aware.** All offset changes, purge, topic delete, ACL revoke, and broad ACL grants are pinned R3; wildcard/glob targets escalate; an unknown operation fails closed to the highest tier — it never falls to R0.
 3. **🤖 AI agents must never invent `--ticket`, `--allow-*`, or a high-risk `--yes`.** Those are *human* authorization inputs. An agent should surface "this needs approval X" to its operator and stop.
 
 ---
@@ -179,14 +186,89 @@ Offsets are a Kafka and Pulsar concept. On RabbitMQ and RocketMQ, `group lag` /
 </details>
 
 <details>
-<summary><b>message</b> — peek & produce</summary>
+<summary><b>message</b> — peek, tail & produce</summary>
 
 ```bash
 mqgov message peek    <topic> [--partition N] [--offset N] [--count N] -o json     # R0, non-destructive, fingerprints only
+mqgov message tail    <topic> [--partition N] [--from earliest|latest|offset:N] [--follow] [--max-messages N] [--timeout 30s] -o json
 mqgov message produce <topic> [--key <k>] [--body <text>] --yes                    # R1 (R3 + --allow-internal-produce for internal topics)
 ```
 
-`peek` never consumes a message or moves a cursor, and returns only sha256 fingerprints (`keySha256`, `bodySha256`, size) — never the body. On RocketMQ, `peek` fails closed (`NOT_IMPLEMENTED`).
+`peek` and `tail` never consume a message or move a cursor, and return only sha256 fingerprints (`keySha256`, `bodySha256`, size, optional timestamp) — never the body. `tail` is bounded by `--max-messages` and `--timeout`; `--follow` streams new messages only until those bounds or cancellation. Tail is supported by Kafka and Pulsar. On RabbitMQ and RocketMQ, `tail` fails closed (`NOT_IMPLEMENTED`); on RocketMQ, `peek` also fails closed.
+</details>
+
+<details>
+<summary><b>dlq</b> — dead-letter queue governance</summary>
+
+```bash
+mqgov dlq list [--topic <source-or-dlq>] [--group <group-or-sub>] [--pattern <name|glob>] -o json     # R0
+mqgov dlq peek <dlq> [--topic <source>] [--group <group-or-sub>] [--count N] -o json                   # R0, fingerprints only
+mqgov dlq redrive <dlq> --target <live-topic> [--count N] --dry-run -o json                            # R0 preview
+mqgov dlq redrive <dlq> --target <live-topic> [--count N] --yes --ticket <t> --allow-internal-produce  # R3
+mqgov dlq purge <dlq> --dry-run -o json                                                                 # R0 preview
+mqgov dlq purge <dlq> --yes --ticket <t> --allow-topic-purge                                           # R3
+```
+
+DLQ mapping is backend-native and honest: RocketMQ lists `%DLQ%{consumerGroup}` topics only; RabbitMQ treats DLQs as ordinary queues fed by a dead-letter exchange; Kafka has no native DLQ and never auto-discovers one, so use an explicit DLQ topic for peek/redrive/purge; Pulsar uses `{topic}-{subscription}-DLQ`. Unsupported verbs return `NOT_IMPLEMENTED`.
+
+Redrive is governed as internal produce: dry-run is a read-only preview and real execution requires `--allow-internal-produce`. Audit remains fingerprint/count-only and never stores message body, key, or headers.
+</details>
+
+<details>
+<summary><b>schema</b> — schema registry</summary>
+
+```bash
+mqgov schema list [--pattern <subject>] -o json
+mqgov schema describe <subject-or-topic> [--version latest|N] -o json
+mqgov schema check <subject-or-topic> --schema-file ./next.avsc --schema-type AVRO [--version latest] -o json
+```
+
+`schema list`, `schema describe`, and `schema check` are R0 and audited. `check` uses read-only compatibility endpoints and never registers, deletes, or evolves a schema. Kafka maps to Confluent Schema Registry (`GET /subjects`, `GET /subjects/{subject}/versions`, `GET /subjects/{subject}/versions/{version|latest}`, and `POST /compatibility/subjects/{subject}/versions/{version}`). Pulsar maps to its built-in admin schema endpoints under `/admin/v2/schemas/{tenant}/{namespace}/{topic}`. RabbitMQ and RocketMQ fail closed with `NOT_IMPLEMENTED`. Audit stores only subject/version metadata and schema hashes, never schema text or registry credentials.
+</details>
+
+<details>
+<summary><b>fleet</b> — cross-context read-only views</summary>
+
+```bash
+mqgov fleet status --all -o json
+mqgov fleet topics --contexts dev,staging --pattern orders -o json
+```
+
+`fleet status` fans out `Ping`, `Describe`, and `Capabilities` across selected contexts. `fleet topics` fans out topic listing and tags every row with its source context. Select contexts with exactly one of `--all` or `--contexts a,b,c`. Fleet is R0 only: each per-context read still runs through the same R0 classification and authorization path as a single-context command, using that context's own stored credentials. Partial failures are reported per context as `denied`, `unreachable`, or `error` data and the command still exits 0.
+</details>
+
+<details>
+<summary><b>acl</b> — broker access control</summary>
+
+```bash
+mqgov acl list [--principal <P>] [--resource-type <T>] [--resource-name <N>] -o json
+
+# Kafka broker ACLs
+mqgov acl grant --principal User:svc --resource-type topic --resource-name orders \
+  --pattern literal --operation read --permission allow --yes --ticket <t>
+
+mqgov acl revoke --principal User:svc --resource-type topic --resource-name orders \
+  --pattern literal --operation read --permission allow \
+  --yes --ticket <t> --allow-destructive-acl
+
+# RabbitMQ native user-vhost permissions
+mqgov acl grant --principal svc --vhost / --resource-type vhost --resource-name '^orders$' \
+  --pattern regex --operation read --permission allow --yes --ticket <t>
+
+mqgov acl revoke --principal svc --vhost / --resource-type vhost --resource-name '^orders$' \
+  --pattern regex --operation read --permission allow \
+  --yes --ticket <t> --allow-destructive-acl
+
+# Pulsar native namespace/topic permissions
+mqgov acl grant --principal app-role --resource-type namespace --resource-name public/default \
+  --pattern literal --operation produce --permission allow --yes --ticket <t>
+
+mqgov acl revoke --principal app-role --resource-type topic --resource-name orders \
+  --pattern literal --operation consume --permission allow \
+  --yes --ticket <t> --allow-destructive-acl
+```
+
+`acl list` is R0 and audited. Normal `acl grant` is R2. Broad grants (Kafka prefixed pattern, wildcard principal, wildcard resource, cluster resource, `all`, `alter`, cluster-action style operations, broad RabbitMQ regexes such as `.*`, `.+`, `.`, and `orders.*`, or Pulsar `functions`/`sources`/`sinks`/`packages`) and every `acl revoke` are R3 and require `--allow-destructive-acl`. Kafka implements broker ACLs with `literal`/`prefixed` patterns. RabbitMQ maps ACLs to native per-user, per-vhost permission regexes (`configure`, `write`, `read`) and only supports `--permission allow` with `--pattern regex`. Pulsar maps ACLs to native role permissions on namespaces or topics with actions `produce`, `consume`, `functions`, `sources`, `sinks`, and `packages`; it is allow-only and uses `--pattern literal`. RocketMQ fails closed with `NOT_IMPLEMENTED`: broker ACLs are managed through broker-side `plain_acl.yml` or the official Java `mqadmin`, because `rocketmq-client-go/v2` exposes no clean public ACL admin API.
 </details>
 
 <details>
@@ -194,12 +276,12 @@ mqgov message produce <topic> [--key <k>] [--body <text>] --yes
 
 ```bash
 # Backend-bound contexts (credentials go through credstore, never plaintext)
-mqgov ctx set <name> --backend kafka    --brokers <h:p,h:p> [--sasl-mechanism PLAIN] [--tls --ca-cert <f>] [--protected]
+mqgov ctx set <name> --backend kafka    --brokers <h:p,h:p> [--sasl-mechanism PLAIN] [--tls --ca-cert <f>] [--schema-registry-url <url>] [--schema-registry-username <u>] [--schema-registry-password <p>] [--protected]
 mqgov ctx set <name> --backend rabbitmq (--amqp-url <url> | --host <h> --port <p> --vhost </>) --management-url <url>
 mqgov ctx set <name> --backend pulsar   --service-url pulsar://<h:p> --admin-url http://<h:p> [--tenant public] [--pulsar-namespace default]
 mqgov ctx set <name> --backend rocketmq --nameservers <h:p,h:p> [--broker-addr <h:p>]
 mqgov ctx use|list|current|delete|test
-#   secrets: --password <pw|token|secretKey> --credential-backend <encrypted-file|keychain|...>  (a non-plain backend is required)
+#   secrets: --password <pw|token|secretKey> and --schema-registry-password <pw> go through --credential-backend <encrypted-file|keychain|...>  (a non-plain backend is required)
 
 # Audit (tamper-evident, fingerprint-only)
 mqgov audit query  [--since 24h] [--type <t>] [--operator <o>] [--status <s>] [--limit 100] -o json
@@ -220,7 +302,7 @@ mqgov version
 
 mqgov-cli is designed to be driven by autonomous agents safely:
 
-- Run `mqgov capabilities -o json` first to discover what the bound backend supports — brokers differ, don't assume (e.g. RabbitMQ/RocketMQ have no offsets; RocketMQ has no peek).
+- Run `mqgov capabilities -o json` first to discover what the bound backend supports — brokers differ, don't assume (e.g. Kafka, RabbitMQ, and Pulsar support `acl` with different native models; Kafka and Pulsar support `schema`; RabbitMQ/RocketMQ have no offsets, schema registry, or tail; RocketMQ has no peek). Use `fleet status --all -o json` for a read-only cross-context dashboard.
 - Use `-o json` everywhere; every command returns a stable, versioned envelope.
 - Get blast radius from `--dry-run` / `--plan`, never from your own reasoning.
 - **Never self-fill `--ticket`, `--allow-*`, or a high-risk `--yes`.** Surface the required human approval and stop.

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "mqgov-cli",
-  "version": "0.1.0",
+  "version": "0.3.0",
   "description": "Governed message-broker operations CLI for AI agents (Kafka, RabbitMQ, Pulsar, RocketMQ)",
   "bin": {
     "mqgov": "bin/mqgov-cli.js",