mqgov-cli 0.2.0 → 0.4.0

package/README.md

@@ -4,7 +4,7 @@
 
 **Governed message-broker operations for humans _and_ AI agents.**
 
-One safe command line for **Kafka**, **RabbitMQ**, **Pulsar**, and **RocketMQ** — list, describe, peek, tail, produce, reset offsets, inspect ACLs, purge, and delete topics without ever fat-fingering production or silently draining a queue.
+One safe command line for **Kafka**, **RabbitMQ**, **Pulsar**, and **RocketMQ** — list, describe, peek, tail, produce, govern DLQs, reset offsets, inspect ACLs and schemas, purge, and delete topics without ever fat-fingering production or silently draining a queue.
 
 [![npm version](https://img.shields.io/npm/v/mqgov-cli.svg)](https://www.npmjs.com/package/mqgov-cli)
 [![CI](https://github.com/JiangHe12/mqgov-cli/actions/workflows/ci.yml/badge.svg)](https://github.com/JiangHe12/mqgov-cli/actions/workflows/ci.yml)
@@ -38,10 +38,10 @@ It's built on the shared [`opskit-core`](https://github.com/JiangHe12/opskit-cor
 | | |
 |---|---|
 | 📨 **Four brokers** | **Kafka** (franz-go), **RabbitMQ** (AMQP + management API), **Pulsar** (client + admin REST), **RocketMQ** (rocketmq-client-go/v2). One backend-agnostic governance model; pick per context or override per command. |
-| 🧱 **topic / group / message / acl** | topics: list · describe · create · alter · delete · purge. consumer groups: list · lag · reset-offset. messages: non-destructive peek · tail · produce. ACLs: list · grant · revoke where supported. |
+| 🧱 **topic / group / message / dlq / acl / schema / fleet** | topics: list · describe · create · alter · delete · purge. consumer groups: list · lag · reset-offset. messages: non-destructive peek · tail · bounded mirror · produce. DLQs: list · peek · redrive · purge through native broker models. ACLs: list · grant · revoke where supported. Schemas: list · describe · check · register · delete where native schema registry support exists. Fleet: read-only status and topic inventory across configured contexts. |
 | 🔐 **R0–R3 governance** | every operation is risk-classified by the fail-closed `mqclass` engine; protected contexts and internal/system topics escalate one tier; AI callers can never self-authorize. |
 | 🎯 **Real blast-radius preview** | `reset-offset --dry-run` and `purge --dry-run` compute the actual per-partition message delta from the live broker — no guessing. The preview is read-only and never mutates. |
-| 👀 **Non-destructive peek/tail** | inspect or stream messages as fingerprints without consuming them or moving any cursor (Kafka direct partition reads, Pulsar Reader, RabbitMQ get+requeue for peek only). Where a broker can't guarantee this, the operation fails closed rather than silently consuming. |
+| 👀 **Non-destructive peek/tail/mirror source** | inspect, stream, or bounded-copy messages without consuming them or moving any cursor where the broker can guarantee it (Kafka direct reads, Pulsar Reader). Where a broker can't guarantee this, the operation fails closed rather than silently consuming. |
 | 🧭 **Honest capabilities** | brokers differ — mqgov reports what each one actually supports (`capabilities -o json`) and **fails closed with `NOT_IMPLEMENTED`** for the rest, never faking it. |
 | 📜 **Tamper-evident audit** | hash-chained log of every action (sha256 fingerprints + counts, **no message bodies/keys/headers**); `audit verify` detects tampering. |
 | 🩺 **Ops & DX** | backend-bound `ctx` contexts with credstore-backed secrets, `doctor` diagnostics, shell `completion`, OpenTelemetry traces/metrics, JSON output everywhere. |
@@ -58,10 +58,14 @@ It's built on the shared [`opskit-core`](https://github.com/JiangHe12/opskit-cor
 | **offset lag / reset** | ✅ | ✅ (cursor) | ❌ (no offsets) | ❌ |
 | alter partitions | ✅ | ✅ | ❌ | ❌ |
 | purge | ✅ | ✅ | ✅ | ❌ |
-| **ACL list / grant / revoke** | ✅ | ✅ namespace/topic permissions | ✅ user-vhost permissions | ❌ `NOT_IMPLEMENTED` |
+| **DLQ list / peek / redrive / purge** | list ❌; explicit topic peek/redrive/purge ✅ | ✅ `{topic}-{subscription}-DLQ` | ✅ DLX queues | list ✅ `%DLQ%group`; others ❌ |
+| **ACL list / grant / revoke** | ✅ | ✅ namespace/topic permissions | ✅ user-vhost permissions | ❌ `NOT_IMPLEMENTED`³ |
+| **schema list / describe / check / register / delete** | ✅ Confluent Schema Registry | ✅ built-in admin schema API | ❌ `NOT_IMPLEMENTED` | ❌ `NOT_IMPLEMENTED` |
 
 ¹ RocketMQ's Go v2 `PullConsumer` enters the consumer-group lifecycle and commits offsets, so it cannot guarantee non-destructive peek/tail — mqgov fails closed instead of silently advancing offsets. ² RabbitMQ has no forward non-destructive tail because reads are consume/requeue oriented. Unsupported operations always return `NOT_IMPLEMENTED` (exit 12), never a fake success.
 
+³ RocketMQ broker ACLs live in broker-side `plain_acl.yml`, but `rocketmq-client-go/v2` does not expose a public, cgo-free admin API for reading or changing that config. mqgov does not shell out to the Java `mqadmin` tool and does not hand-roll remoting commands; manage RocketMQ ACLs out of band with broker configuration or official mqadmin until the Go client exposes a clean API.
+
 ---
 
 ## 📦 Install
@@ -124,12 +128,12 @@ Every command is sorted into one of four **risk tiers** by the fail-closed `mqcl
 
 | Tier | What it covers | What you must provide |
 |:---:|---|---|
-| **R0** | Reads & previews (`topic list/describe`, `group list/lag`, `message peek`, `message tail`, `acl list`, `*-dry-run`, `audit query/verify`, `doctor`) | Nothing — but it's still audited |
-| **R1** | Ordinary writes (`message produce`, `topic create`) | `--yes` (or an interactive confirmation) |
-| **R2** | Elevated mutations (`topic alter`, `group create/delete`, `acl grant`, produce to a **protected** topic) | `--yes` **and** a non-empty `--ticket` |
-| **R3** | Destructive / irreversible (`group reset-offset`, `topic purge`, `topic delete`, broad `acl grant`, `acl revoke`, produce to an **internal/system** topic) | The above **plus** the exact `--allow-*` flag |
+| **R0** | Reads & previews (`topic list/describe`, `group list/lag`, `message peek`, `message tail`, `dlq list/peek`, `acl list`, `schema list/describe/check`, `fleet status/topics`, `*-dry-run`, `audit query/verify`, `doctor`) | Nothing — but it's still audited |
+| **R1** | Ordinary writes (`message produce`, target side of `message mirror`, `topic create`, `schema register` for a new subject) | `--yes` (or an interactive confirmation) |
+| **R2** | Elevated mutations (`topic alter`, `group create/delete`, `acl grant`, `schema register` for an existing subject, produce/mirror to a **protected** topic) | `--yes` **and** a non-empty `--ticket` |
+| **R3** | Destructive / irreversible (`group reset-offset`, `topic purge`, `topic delete`, `schema delete`, `dlq redrive`, `dlq purge`, broad `acl grant`, `acl revoke`, produce/mirror to an **internal/system** topic) | The above **plus** the exact `--allow-*` flag |
 
-The R3 allow flags: `--allow-offset-reset`, `--allow-topic-purge`, `--allow-topic-delete`, `--allow-destructive-acl`, `--allow-internal-produce`.
+The R3 allow flags: `--allow-offset-reset`, `--allow-topic-purge`, `--allow-topic-delete`, `--allow-destructive-acl`, `--allow-internal-produce`, `--allow-schema-delete`.
 
 **Protected contexts, protected topics, and internal/system topics raise the tier by one.** For example, producing to `__consumer_offsets` is treated as a destructive R3 operation and needs `--allow-internal-produce`.
 
@@ -182,15 +186,62 @@ Offsets are a Kafka and Pulsar concept. On RabbitMQ and RocketMQ, `group lag` /
 </details>
 
 <details>
-<summary><b>message</b> — peek, tail & produce</summary>
+<summary><b>message</b> — peek, tail, mirror & produce</summary>
 
 ```bash
 mqgov message peek    <topic> [--partition N] [--offset N] [--count N] -o json     # R0, non-destructive, fingerprints only
 mqgov message tail    <topic> [--partition N] [--from earliest|latest|offset:N] [--follow] [--max-messages N] [--timeout 30s] -o json
+mqgov message mirror  <source-topic> --to-context <ctx> --to-topic <topic> --limit 100 --dry-run -o json
+mqgov message mirror  <source-topic> --to-context <ctx> --to-topic <topic> --limit 100 --yes -o json
 mqgov message produce <topic> [--key <k>] [--body <text>] --yes                    # R1 (R3 + --allow-internal-produce for internal topics)
 ```
 
-`peek` and `tail` never consume a message or move a cursor, and return only sha256 fingerprints (`keySha256`, `bodySha256`, size, optional timestamp) — never the body. `tail` is bounded by `--max-messages` and `--timeout`; `--follow` streams new messages only until those bounds or cancellation. Tail is supported by Kafka and Pulsar. On RabbitMQ and RocketMQ, `tail` fails closed (`NOT_IMPLEMENTED`); on RocketMQ, `peek` also fails closed.
+`peek` and `tail` never consume a message or move a cursor, and return only sha256 fingerprints (`keySha256`, `bodySha256`, size, optional timestamp) — never the body. `tail` is bounded by `--max-messages` and `--timeout`; `--follow` streams new messages only until those bounds or cancellation.
+
+`message mirror` is a bounded one-shot copy, never a daemon. It performs two independent authorizations: a source-side non-destructive read against the source context, then a target-side produce against `--to-context`. `--dry-run` / `--plan` is an R0 preview that reads/counts but does not produce. Kafka and Pulsar can be mirror sources; RabbitMQ and RocketMQ source mirroring fail closed with `NOT_IMPLEMENTED` because their available read APIs cannot guarantee non-destructive full-message reads. Keys, bodies, and headers flow only in process memory; audit records source/target/count and body sha256 aggregation only. Kafka supports `--from earliest|latest|offset:N|timestamp:<RFC3339>` and `--partition`; Pulsar supports `earliest|latest|timestamp:<RFC3339>` and all-partition reads. Headers are copied where both backends can express string/byte headers; unsupported source concepts are not fabricated.
+</details>
+
+<details>
+<summary><b>dlq</b> — dead-letter queue governance</summary>
+
+```bash
+mqgov dlq list [--topic <source-or-dlq>] [--group <group-or-sub>] [--pattern <name|glob>] -o json     # R0
+mqgov dlq peek <dlq> [--topic <source>] [--group <group-or-sub>] [--count N] -o json                   # R0, fingerprints only
+mqgov dlq redrive <dlq> --target <live-topic> [--count N] --dry-run -o json                            # R0 preview
+mqgov dlq redrive <dlq> --target <live-topic> [--count N] --yes --ticket <t> --allow-internal-produce  # R3
+mqgov dlq purge <dlq> --dry-run -o json                                                                 # R0 preview
+mqgov dlq purge <dlq> --yes --ticket <t> --allow-topic-purge                                           # R3
+```
+
+DLQ mapping is backend-native and honest: RocketMQ lists `%DLQ%{consumerGroup}` topics only; RabbitMQ treats DLQs as ordinary queues fed by a dead-letter exchange; Kafka has no native DLQ and never auto-discovers one, so use an explicit DLQ topic for peek/redrive/purge; Pulsar uses `{topic}-{subscription}-DLQ`. Unsupported verbs return `NOT_IMPLEMENTED`.
+
+Redrive is governed as internal produce: dry-run is a read-only preview and real execution requires `--allow-internal-produce`. Audit remains fingerprint/count-only and never stores message body, key, or headers.
+</details>
+
+<details>
+<summary><b>schema</b> — schema registry</summary>
+
+```bash
+mqgov schema list [--pattern <subject>] -o json
+mqgov schema describe <subject-or-topic> [--version latest|N] -o json
+mqgov schema check <subject-or-topic> --schema-file ./next.avsc --schema-type AVRO [--version latest] -o json
+mqgov schema register <subject-or-topic> --schema-file ./next.avsc --schema-type AVRO --yes -o json
+mqgov schema register <subject-or-topic> --schema-file ./next.avsc --schema-type AVRO --yes --ticket <t> -o json
+mqgov schema delete <subject-or-topic> [--version N] [--permanent] --yes --ticket <t> --allow-schema-delete -o json
+```
+
+`schema list`, `schema describe`, and `schema check` are R0 and audited. `schema register` is R1 for a new subject and R2 when the subject already exists; registering a new version is the evolution path. Existing subjects first run the backend compatibility check and incompatible schemas are rejected before registration. `schema delete` is R3 and requires `--allow-schema-delete`. Kafka maps to Confluent Schema Registry, including soft delete and hard delete with `--permanent`. Pulsar maps to its built-in admin schema endpoints; because Pulsar has no soft/hard split, this backend only accepts permanent subject deletion and returns `NOT_IMPLEMENTED` for soft or version delete. RabbitMQ and RocketMQ fail closed with `NOT_IMPLEMENTED`. Audit stores only subject/version metadata and schema hashes, never schema text or registry credentials.
+</details>
+
+<details>
+<summary><b>fleet</b> — cross-context read-only views</summary>
+
+```bash
+mqgov fleet status --all -o json
+mqgov fleet topics --contexts dev,staging --pattern orders -o json
+```
+
+`fleet status` fans out `Ping`, `Describe`, and `Capabilities` across selected contexts. `fleet topics` fans out topic listing and tags every row with its source context. Select contexts with exactly one of `--all` or `--contexts a,b,c`. Fleet is R0 only: each per-context read still runs through the same R0 classification and authorization path as a single-context command, using that context's own stored credentials. Partial failures are reported per context as `denied`, `unreachable`, or `error` data and the command still exits 0.
 </details>
 
 <details>
@@ -224,7 +275,7 @@ mqgov acl revoke --principal app-role --resource-type topic --resource-name orde
   --yes --ticket <t> --allow-destructive-acl
 ```
 
-`acl list` is R0 and audited. Normal `acl grant` is R2. Broad grants (Kafka prefixed pattern, wildcard principal, wildcard resource, cluster resource, `all`, `alter`, cluster-action style operations, broad RabbitMQ regexes such as `.*`, `.+`, `.`, and `orders.*`, or Pulsar `functions`/`sources`/`sinks`/`packages`) and every `acl revoke` are R3 and require `--allow-destructive-acl`. Kafka implements broker ACLs with `literal`/`prefixed` patterns. RabbitMQ maps ACLs to native per-user, per-vhost permission regexes (`configure`, `write`, `read`) and only supports `--permission allow` with `--pattern regex`. Pulsar maps ACLs to native role permissions on namespaces or topics with actions `produce`, `consume`, `functions`, `sources`, `sinks`, and `packages`; it is allow-only and uses `--pattern literal`. RocketMQ fails closed with `NOT_IMPLEMENTED`.
+`acl list` is R0 and audited. Normal `acl grant` is R2. Broad grants (Kafka prefixed pattern, wildcard principal, wildcard resource, cluster resource, `all`, `alter`, cluster-action style operations, broad RabbitMQ regexes such as `.*`, `.+`, `.`, and `orders.*`, or Pulsar `functions`/`sources`/`sinks`/`packages`) and every `acl revoke` are R3 and require `--allow-destructive-acl`. Kafka implements broker ACLs with `literal`/`prefixed` patterns. RabbitMQ maps ACLs to native per-user, per-vhost permission regexes (`configure`, `write`, `read`) and only supports `--permission allow` with `--pattern regex`. Pulsar maps ACLs to native role permissions on namespaces or topics with actions `produce`, `consume`, `functions`, `sources`, `sinks`, and `packages`; it is allow-only and uses `--pattern literal`. RocketMQ fails closed with `NOT_IMPLEMENTED`: broker ACLs are managed through broker-side `plain_acl.yml` or the official Java `mqadmin`, because `rocketmq-client-go/v2` exposes no clean public ACL admin API.
 </details>
 
 <details>
@@ -232,12 +283,12 @@ mqgov acl revoke --principal app-role --resource-type topic --resource-name orde
 
 ```bash
 # Backend-bound contexts (credentials go through credstore, never plaintext)
-mqgov ctx set <name> --backend kafka    --brokers <h:p,h:p> [--sasl-mechanism PLAIN] [--tls --ca-cert <f>] [--protected]
+mqgov ctx set <name> --backend kafka    --brokers <h:p,h:p> [--sasl-mechanism PLAIN] [--tls --ca-cert <f>] [--schema-registry-url <url>] [--schema-registry-username <u>] [--schema-registry-password <p>] [--protected]
 mqgov ctx set <name> --backend rabbitmq (--amqp-url <url> | --host <h> --port <p> --vhost </>) --management-url <url>
 mqgov ctx set <name> --backend pulsar   --service-url pulsar://<h:p> --admin-url http://<h:p> [--tenant public] [--pulsar-namespace default]
 mqgov ctx set <name> --backend rocketmq --nameservers <h:p,h:p> [--broker-addr <h:p>]
 mqgov ctx use|list|current|delete|test
-#   secrets: --password <pw|token|secretKey> --credential-backend <encrypted-file|keychain|...>  (a non-plain backend is required)
+#   secrets: --password <pw|token|secretKey> and --schema-registry-password <pw> go through --credential-backend <encrypted-file|keychain|...>  (a non-plain backend is required)
 
 # Audit (tamper-evident, fingerprint-only)
 mqgov audit query  [--since 24h] [--type <t>] [--operator <o>] [--status <s>] [--limit 100] -o json
@@ -258,7 +309,7 @@ mqgov version
 
 mqgov-cli is designed to be driven by autonomous agents safely:
 
-- Run `mqgov capabilities -o json` first to discover what the bound backend supports — brokers differ, don't assume (e.g. Kafka, RabbitMQ, and Pulsar support `acl` with different native models; RabbitMQ/RocketMQ have no offsets; RabbitMQ/RocketMQ have no tail; RocketMQ has no peek).
+- Run `mqgov capabilities -o json` first to discover what the bound backend supports — brokers differ, don't assume (e.g. Kafka, RabbitMQ, and Pulsar support `acl` with different native models; Kafka and Pulsar support `schema`; RabbitMQ/RocketMQ have no offsets, schema registry, or tail; RocketMQ has no peek). Use `fleet status --all -o json` for a read-only cross-context dashboard.
 - Use `-o json` everywhere; every command returns a stable, versioned envelope.
 - Get blast radius from `--dry-run` / `--plan`, never from your own reasoning.
 - **Never self-fill `--ticket`, `--allow-*`, or a high-risk `--yes`.** Surface the required human approval and stop.

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "mqgov-cli",
-  "version": "0.2.0",
+  "version": "0.4.0",
   "description": "Governed message-broker operations CLI for AI agents (Kafka, RabbitMQ, Pulsar, RocketMQ)",
   "bin": {
     "mqgov": "bin/mqgov-cli.js",