omq-cli 0.13.0 → 0.14.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 0135d1ab507a4e82186cae115a477ccbd4b231e12a7450cb8989a5c0c1479e50
-  data.tar.gz: efc832159dc103ec4b18ba5f65acb5976a09c7530b246e5e433795357ca21181
+  metadata.gz: b5a558fd06799fcf4bee5d5036a4cac86c8482554df3998e5f42171d04e520db
+  data.tar.gz: e53bb5236bb5bc660e04ad00c10aa0f3b1e4a848ed9d74363c3943f877bf210f
 SHA512:
-  metadata.gz: a8093420e5209f9e4672dd15a343c60167959532d90fda0fce7f108042807a65d27f033eb39a3abfee80059e8832d2e90680a624f36575c8b608f2da5095865c
-  data.tar.gz: 30dc24c50221b726130e638764d891d2736044b358b364f2bd987a21eceac85bea44ed0821ab601d1070d2e60bc10492a61ffd8baa8129588a0dce06b72ec192
+  metadata.gz: 5841f42d980388be4ce4d0feea88b76a358e86f6ba6e64f6b1775dfdcad16dc349ba3866c824fa74ffb88005876e64c027de9fd93fa21417c8a5a53be3180861
+  data.tar.gz: 4b68d2c56882c87a06e75fa348eff4f9df75d5cd0c3a85c26e20e6676780c61c0413e329dff7bdbb082b861795cdf09bb30aa4b61c7279c88bbd2e29373d1552

data/CHANGELOG.md

@@ -1,5 +1,86 @@
 # Changelog
 
+## 0.14.0 — 2026-04-13
+
+### Added
+
+- **Receive-capable sockets decompress by default.** All socket types
+  except pure senders (`push`, `pub`, `scatter`, `radio`) now advertise
+  the ZMTP-Zstd profile in **passive mode** at startup, so they accept
+  compressed frames from any active-sender peer without requiring
+  `-z` on the receive side. They never compress their own outgoing
+  frames in this mode — use `-z` / `-Z` / `--compress=LEVEL` on the
+  sender to opt it in. A `push` piped into a `pull` with no flags on
+  either side stays uncompressed; `omq push -z | omq pull` compresses
+  on the wire and the pull side decodes transparently. This is the
+  RFC Sec. 6.4 "Passive senders" mode; requires omq-rfc-zstd >= 0.1.0.
+- **`-Z` flag for better-ratio compression (zstd level 3).** `-z`
+  remains the fast default (level -3) and `--compress=LEVEL` takes
+  a custom zstd level (e.g. `--compress=19`, `--compress=-1`). Short
+  bundling (`-zvvv`, `-Zvvv`) still works.
+- **`-vvv` logs `ZDICT` exchange.** When the auto-trained dictionary
+  is shipped/received, the trace prints `>> ZDICT (NB)` on the sender
+  and `<< ZDICT (NB)` on the receiver.
+- **`-vvv` wire-size annotation for compressed traces.** Message
+  previews on compressed sockets include the post-compression byte
+  count: `(280B wire=29B) ZZ…`. Plumbed from the ZMTP-Zstd wrapper
+  through the engine's verbose monitor.
+
+### Changed
+
+- **Compression backend switched from `rlz4` to `omq-rfc-zstd`.**
+  Compression is now a ZMTP wire-protocol extension negotiated via
+  the `X-Compression` READY property and applied below the
+  application API. Auto-trained dictionaries are shipped over a
+  `ZDICT` command frame once the sender has enough samples. The
+  `Formatter` no longer compresses or decompresses anything — it
+  only encodes/decodes wire formats. Pipe `-z` is no longer modal
+  (`compress_in`/`compress_out` removed) since compression is a
+  per-socket, send-side property negotiated with each peer.
+- **`-vvv` output ordering under compression.** At `-vvv`, the
+  monitor fiber now writes both the trace line and the plaintext
+  body, so trace-and-body pairs land on the tty in order instead of
+  interleaving between the recv pump and the app fiber.
+- **TCP host normalization moved into `OMQ::Transport::TCP`.** `omq`
+  v0.19.0 now handles `tcp://*:PORT`, `tcp://:PORT`, and
+  `tcp://localhost:PORT` natively (including dual-stack `*` binding
+  both IPv4 and IPv6 wildcards), so `CliParser` no longer rewrites
+  these URLs before handing them off. Removed
+  `CliParser.loopback_bind_host` and the `normalize_bind`/
+  `normalize_connect`/`normalize_ep` block. Requires `omq ~> 0.19`.
+- **Terminate on protocol errors instead of silent reconnect.** When
+  a peer sends a frame that violates the ZMTP wire protocol
+  (oversized, bad framing, zstd bytebomb, nonce exhaustion, …), the
+  library drops that one connection and reconnects — the libzmq
+  parity behavior. The CLI is a different audience: a persistent
+  protocol violation is almost always a misconfiguration the user
+  needs to see, not silently paper over. Every runner
+  (`BaseRunner`, `PipeRunner`, `ParallelWorker`, `PipeWorker`) now
+  attaches a monitor that watches for `:disconnected` events whose
+  `detail[:error]` is a `Protocol::ZMTP::Error`, prints
+  `omq: <reason>` to stderr, kills the socket, and exits with
+  status 1. Requires `omq ~> 0.19.2` for the new `:disconnected`
+  detail shape and `Socket#engine` accessor.
+- **`-vvv` disconnect events render the reason in parentheses.**
+  `Term.format_event` now pretty-prints `:disconnected` details
+  that contain a `:reason` key, e.g.
+  `disconnected tcp://:5555 (frame size 1024 exceeds max_message_size 32)`,
+  instead of dumping the raw hash.
+
+### Fixed
+
+- **`--recv-maxsz` is now actually applied in pipe and parallel
+  modes.** `PipeRunner`, `PipeWorker`, and `ParallelWorker` were
+  only calling `SocketSetup.apply_options` + `apply_compression`
+  and skipping `max_message_size` entirely — so the default 1 MiB
+  cap (and any `--recv-maxsz` override) silently had no effect on
+  `omq pipe` or `omq pull -P`. Extracted the logic into
+  `SocketSetup.apply_recv_maxsz` and wired it into all four setup
+  paths (sequential pull/rep, sequential pipe, parallel worker,
+  pipe worker). Oversized frames now drop the connection as
+  intended and — combined with the CLI termination policy above —
+  exit with a clear error instead of hanging in a reconnect loop.
+
 ## 0.13.0 — 2026-04-12
 
 ### Added

data/README.md

@@ -348,8 +348,16 @@ omq pull -b tcp://:5557 -t 5
 
 ## Compression
 
-Both sides must use `--compress` (`-z`). Uses LZ4 frame format, provided
-by the `rlz4` gem (Ractor-safe, Rust extension via `lz4_flex`).
+Set `--compress` (`-z`) on either or both sides. The flag enables
+ZMTP-Zstd (provided by `omq-rfc-zstd`), a wire-protocol extension
+that negotiates Zstandard compression during the ZMTP handshake via
+an `X-Compression` READY metadata field. If both peers advertise it,
+each side compresses its outgoing frames; if only one side does, the
+connection stays plaintext (no error). The extension uses the
+auto-trained dictionary mode: the sender feeds the first messages
+into a dictionary trainer, ships the trained dictionary over a
+ZDICT command frame, then switches to dict-bound compression for
+the rest of the connection.
 
 ```sh
 omq push -c tcp://remote:5557 -z < data.txt

data/lib/omq/cli/base_runner.rb

@@ -15,7 +15,7 @@ module OMQ
       def initialize(config, socket_class)
         @config = config
         @klass  = socket_class
-        @fmt    = Formatter.new(config.format, compress: config.compress)
+        @fmt    = Formatter.new(config.format)
       end
 
 
@@ -26,13 +26,17 @@ module OMQ
       def call(task)
         set_process_title
         setup_socket
-        start_event_monitor if config.verbose >= 2
+        start_event_monitor
         maybe_start_transient_monitor(task)
         sleep(config.delay) if config.delay && config.recv_only?
         wait_for_peer if needs_peer_wait?
         run_begin_blocks
         run_loop(task)
         run_end_blocks
+      rescue OMQ::SocketDeadError => error
+        reason = error.cause&.message || error.message
+        $stderr.write("omq: #{reason}\n")
+        exit 1
       ensure
         @sock&.close
       end
@@ -313,41 +317,21 @@ module OMQ
 
       def send_msg(parts)
         return if parts.empty?
-        log_parts = parts
         parts = [Marshal.dump(parts)] if config.format == :marshal
-        if config.compress
-          wire = @fmt.compress(parts)
-          trace_msg(">>", log_parts, wire)
-          @sock.send(wire)
-        else
-          @sock.send(parts)
-        end
+        @sock.send(parts)
         transient_ready!
       end
 
 
       def recv_msg
-        raw = @sock.receive
-        return nil if raw.nil?
-        parts = @fmt.decompress(raw)
-        trace_msg("<<", parts, raw) if config.compress
+        parts = @sock.receive
+        return nil if parts.nil?
         parts = Marshal.load(parts.first) if config.format == :marshal
         transient_ready!
         parts
       end
 
 
-      # Writes an explicit -vvv message trace line. Used when compression
-      # is active: the engine-level monitor would see compressed bytes,
-      # so we log post-decompression (recv) / pre-compression (send)
-      # and annotate the compressed wire size.
-      def trace_msg(marker, parts, wire)
-        return unless config.verbose >= 3
-        wire_size = wire.sum(&:bytesize)
-        $stderr.write("#{Term.log_prefix(config.timestamps)}omq: #{marker} #{Formatter.preview(parts, wire_size: wire_size)}\n")
-      end
-
-
       def recv_msg_raw
         msg = @sock.receive
         msg&.dup
@@ -408,6 +392,10 @@ module OMQ
 
       def output(parts)
         return if config.quiet || parts.nil?
+        # At -vvv, the monitor fiber owns both the trace and the body
+        # output (see start_event_monitor). Skipping the app-side write
+        # avoids interleaving between the two fibers on a shared tty.
+        return if config.verbose >= 3
         $stdout.write(@fmt.encode(parts))
         $stdout.flush
       end
@@ -480,17 +468,38 @@ module OMQ
       end
 
 
-      # -vv: log connect/disconnect/retry/timeout events via Socket#monitor
-      # -vvv: also log message sent/received traces
+      # Always attached so protocol-level disconnect events can kill
+      # the socket. Verbose gating lives inside the callback:
+      #   -vv  log connect/disconnect/retry/timeout events
+      #   -vvv also log message sent/received traces
       # --timestamps[=s|ms|us]: prepend UTC timestamps to log lines
       def start_event_monitor
-        # When compress is on, BaseRunner#trace_msg logs post-
-        # decompression, so we suppress the engine-level message trace.
-        trace = config.verbose >= 3 && !config.compress
+        trace        = config.verbose >= 3
+        log_events   = config.verbose >= 2
         @sock.monitor(verbose: trace) do |event|
-          Term.write_event(event, config.timestamps)
+          Term.write_event(event, config.timestamps) if log_events
+          if trace && event.type == :message_received && !config.quiet
+            $stdout.write(@fmt.encode(event.detail[:parts]))
+            $stdout.flush
+          end
+          kill_on_protocol_error(event)
         end
       end
+
+
+      # omq-cli policy: a peer that commits a protocol-level violation
+      # (Protocol::ZMTP::Error — oversized frame, decompression
+      # bytebomb, bad framing, …) is almost certainly a
+      # misconfiguration the user needs to see. Mark the socket dead
+      # so the next receive raises SocketDeadError. The library
+      # itself just drops the connection and keeps serving the
+      # others; this stricter policy is CLI-only.
+      def kill_on_protocol_error(event)
+        return unless event.type == :disconnected
+        error = event.detail && event.detail[:error]
+        return unless error.is_a?(Protocol::ZMTP::Error)
+        @sock.engine.signal_fatal_error(error)
+      end
     end
   end
 end

data/lib/omq/cli/cli_parser.rb

@@ -125,8 +125,12 @@ module OMQ
 
         -- Compression ----------------------------------------------
 
-          # both sides must use --compress/-z
-          omq pull --bind tcp://:5557 --compress &
+          # ZMTP-Zstd is negotiated transparently during the handshake.
+          # Receive-capable sockets (pull, sub, rep, ...) advertise the
+          # profile by default in passive mode: they decode compressed
+          # frames from an active sender but never compress their own
+          # outgoing frames. Use -z / -Z on the sender to opt it in.
+          omq pull --bind tcp://:5557 &
           echo "compressible data" | omq push --connect tcp://localhost:5557 -z
 
         -- CURVE Encryption -----------------------------------------
@@ -225,8 +229,7 @@ module OMQ
         rcvbuf:           nil,
         conflate:         false,
         compress:         false,
-        compress_in:      false,
-        compress_out:     false,
+        compress_level:   nil,
         send_expr:        nil,
         recv_expr:        nil,
         parallel:         nil,
@@ -386,16 +389,17 @@ module OMQ
           o.on("--conflate", "Keep only last message per subscriber (PUB/RADIO)") { opts[:conflate] = true }
 
           o.separator "\nCompression:"
-          o.on("-z", "--compress", "LZ4 compression per frame (modal with --in/--out)") do
-            require "rlz4"
-            case pipe_side
-            when :in
-              opts[:compress_in] = true
-            when :out
-              opts[:compress_out] = true
-            else
-              opts[:compress] = true
-            end
+          o.on("-z", "Zstd compression (level -3, fast)") do
+            opts[:compress] = true
+            opts[:compress_level] = -3
+          end
+          o.on("-Z", "Zstd compression (level 3, better ratio)") do
+            opts[:compress] = true
+            opts[:compress_level] = 3
+          end
+          o.on("--compress=LEVEL", Integer, "Zstd compression with custom level (e.g. 19, -1)") do |v|
+            opts[:compress] = true
+            opts[:compress_level] = v
           end
 
           o.separator "\nProcessing (-e = incoming, -E = outgoing):"
@@ -470,26 +474,9 @@ module OMQ
           opts[:type_name] = type_name.downcase
         end
 
-        # Normalize shorthand hostnames to concrete addresses.
-        #
-        # Binds:    tcp://:PORT  → loopback (::1 if IPv6 available, else 127.0.0.1)
-        #           tcp://*:PORT → 0.0.0.0 (all interfaces, IPv4)
-        #           tcp://0.0.0.0:PORT, tcp://[::]:PORT → pass through
-        #
-        # Connects: tcp://:PORT  → localhost (Happy Eyeballs)
-        #           tcp://*:PORT → localhost
-        #
-        # The hang on macOS (IPv6 connect(2) not getting ECONNREFUSED via
-        # kqueue) is fixed by the connect timeout in Engine::Reconnect.
-        loopback        = self.class.loopback_bind_host
-        normalize_bind    = ->(url) { url.sub(%r{\Atcp://\*:}, "tcp://0.0.0.0:").sub(%r{\Atcp://:}, "tcp://#{loopback}:") }
-        normalize_connect = ->(url) { url.sub(%r{\Atcp://(\*|):}, "tcp://localhost:") }
-        normalize_ep      = ->(ep)  { Endpoint.new(ep.bind? ? normalize_bind.call(ep.url) : normalize_connect.call(ep.url), ep.bind?) }
-        opts[:binds].map!(&normalize_bind)
-        opts[:connects].map!(&normalize_connect)
-        opts[:endpoints].map!(&normalize_ep)
-        opts[:in_endpoints].map!(&normalize_ep)
-        opts[:out_endpoints].map!(&normalize_ep)
+        # Host shorthand (tcp://*:PORT, tcp://:PORT, tcp://localhost:PORT)
+        # is normalized inside OMQ::Transport::TCP — see its
+        # #normalize_bind_host / #normalize_connect_host / #loopback_host.
 
         opts
       end
@@ -592,19 +579,6 @@ module OMQ
       end
 
 
-      # Returns the loopback address for bind normalization.
-      # Prefers IPv6 loopback ([::1]) when the host has at least one
-      # non-loopback, non-link-local IPv6 address, otherwise 127.0.0.1.
-      #
-      def self.loopback_bind_host
-        @loopback_bind_host ||= begin
-          has_ipv6 = ::Socket.getifaddrs.any? { |ifa|
-            addr = ifa.addr
-            addr&.ipv6? && !addr.ipv6_loopback? && !addr.ipv6_linklocal?
-          }
-          has_ipv6 ? "[::1]" : "127.0.0.1"
-        end
-      end
     end
   end
 end

data/lib/omq/cli/client_server.rb

@@ -25,8 +25,7 @@ module OMQ
           parts = recv_msg_raw
           break if parts.nil?
           routing_id = parts.shift
-          body       = @fmt.decompress(parts)
-          break unless handle_server_request(routing_id, body)
+          break unless handle_server_request(routing_id, parts)
           i += 1
           break if n && n > 0 && i >= n
         end
@@ -37,15 +36,15 @@ module OMQ
         if config.recv_expr || @recv_eval_proc
           reply = eval_recv_expr(body)
           output([display_routing_id(routing_id), *(reply || [""])])
-          @sock.send_to(routing_id, @fmt.compress(reply || [""]).first)
+          @sock.send_to(routing_id, (reply || [""]).first)
         elsif config.echo
           output([display_routing_id(routing_id), *body])
-          @sock.send_to(routing_id, @fmt.compress(body).first || "")
+          @sock.send_to(routing_id, body.first || "")
         elsif config.data || config.file || !config.stdin_is_tty
           reply = read_next
           return false unless reply
           output([display_routing_id(routing_id), *body])
-          @sock.send_to(routing_id, @fmt.compress(reply).first || "")
+          @sock.send_to(routing_id, reply.first || "")
         end
         true
       end
@@ -66,7 +65,6 @@ module OMQ
             parts = recv_msg_raw
             break if parts.nil?
             routing_id = parts.shift
-            parts      = @fmt.decompress(parts)
             result = eval_recv_expr([display_routing_id(routing_id), *parts])
             output(result)
             i += 1
@@ -77,7 +75,7 @@ module OMQ
 
 
       def send_to_peer(id, parts)
-        @sock.send_to(id, @fmt.compress(parts).first || "")
+        @sock.send_to(id, parts.first || "")
       end
     end
   end

data/lib/omq/cli/config.rb

@@ -44,8 +44,7 @@ module OMQ
       :rcvbuf,
       :conflate,
       :compress,
-      :compress_in,
-      :compress_out,
+      :compress_level,
       :send_expr,
       :recv_expr,
       :parallel,

data/lib/omq/cli/formatter.rb

@@ -2,17 +2,14 @@
 
 module OMQ
   module CLI
-    # Raised when LZ4 decompression fails.
-    class DecompressError < RuntimeError; end
-
-    # Handles encoding/decoding messages in the configured format,
-    # plus optional LZ4 compression.
+    # Handles encoding/decoding messages in the configured format.
+    # Compression is handled below the application API by ZMTP-Zstd
+    # (omq-rfc-zstd) once enabled via +socket.compression=+; the
+    # formatter sees plaintext frames in both directions.
     class Formatter
       # @param format [Symbol] wire format (:ascii, :quoted, :raw, :jsonl, :msgpack, :marshal)
-      # @param compress [Boolean] whether to apply LZ4 compression per frame
-      def initialize(format, compress: false)
-        @format   = format
-        @compress = compress
+      def initialize(format)
+        @format = format
       end
 
 
@@ -83,32 +80,11 @@ module OMQ
       end
 
 
-      # Compresses each frame with LZ4 if compression is enabled.
-      #
-      # @param parts [Array<String>] message frames
-      # @return [Array<String>] optionally compressed frames
-      def compress(parts)
-        @compress ? parts.map { |p| RLZ4.compress(p) if p } : parts
-      end
-
-
-      # Decompresses each frame with LZ4 if compression is enabled.
-      # nil/empty frames pass through — they were nil before send coercion.
-      #
-      # @param parts [Array<String>] possibly compressed message frames
-      # @return [Array<String>] decompressed frames
-      def decompress(parts)
-        @compress ? parts.map { |p| p && !p.empty? ? RLZ4.decompress(p) : p } : parts
-      rescue RLZ4::DecompressError
-        raise DecompressError, "decompression failed (did the sender use --compress?)"
-      end
-
-
       # Formats message parts for human-readable preview (logging).
-      # When +wire_size+ is given, the header also shows the
-      # compressed on-the-wire size: "(29B wire=12B)".
+      # When +wire_size+ is given (ZMTP-Zstd negotiated), the header
+      # also shows the compressed on-the-wire size: "(29B wire=12B)".
       #
-      # @param parts [Array<String>] message frames (decompressed)
+      # @param parts [Array<String>] plaintext message frames
       # @param wire_size [Integer, nil] compressed bytes on the wire
       # @return [String] truncated preview of each frame joined by |
       def self.preview(parts, wire_size: nil)

data/lib/omq/cli/parallel_worker.rb

@@ -29,8 +29,15 @@ module OMQ
           compile_expr
           run_loop
           run_end_block
-        rescue DecompressError => e
-          @error_port.send(e.message)
+        rescue OMQ::SocketDeadError => error
+          # Socket was killed by a protocol violation on the peer side
+          # (see Engine#signal_fatal_error). Surface the underlying
+          # cause via the log stream and exit cleanly -- the Ractor
+          # completes, consumer threads unblock.
+          reason = error.cause&.message || error.message
+          @log_port.send("omq: #{reason}")
+        rescue => error
+          @error_port.send("#{error.class}: #{error.message}")
         ensure
           @sock&.close
         end
@@ -43,6 +50,8 @@ module OMQ
       def setup_socket
         @sock = @config.ffi ? OMQ.const_get(@socket_sym).new(backend: :ffi) : OMQ.const_get(@socket_sym).new
         OMQ::CLI::SocketSetup.apply_options(@sock, @config)
+        OMQ::CLI::SocketSetup.apply_recv_maxsz(@sock, @config)
+        OMQ::CLI::SocketSetup.apply_compression(@sock, @config, @config.type_name)
         @sock.identity = @config.identity if @config.identity
         OMQ::CLI::SocketSetup.attach_endpoints(@sock, @endpoints, verbose: 0)
       end
@@ -57,26 +66,23 @@ module OMQ
 
 
       def start_monitors
-        return unless @config.verbose >= 2
-        # When compress is on, messages are traced explicitly post-
-        # decompression; suppress engine-level message trace to avoid
-        # showing compressed bytes.
-        trace = @config.verbose >= 3 && !@config.compress
+        trace      = @config.verbose >= 3
+        log_events = @config.verbose >= 2
         @sock.monitor(verbose: trace) do |event|
-          @log_port.send(OMQ::CLI::Term.format_event(event, @config.timestamps))
+          @log_port.send(OMQ::CLI::Term.format_event(event, @config.timestamps)) if log_events
+          kill_on_protocol_error(event)
         end
       end
 
 
-      def trace_in(parts, wire)
-        return unless @config.verbose >= 3 && @config.compress
-        @log_port.send("#{OMQ::CLI::Term.log_prefix(@config.timestamps)}omq: << #{OMQ::CLI::Formatter.preview(parts, wire_size: wire.sum(&:bytesize))}")
-      end
-
-
-      def trace_out(parts, wire)
-        return unless @config.verbose >= 3 && @config.compress
-        @log_port.send("#{OMQ::CLI::Term.log_prefix(@config.timestamps)}omq: >> #{OMQ::CLI::Formatter.preview(parts, wire_size: wire.sum(&:bytesize))}")
+      # Mirrors BaseRunner#kill_on_protocol_error: CLI-level policy
+      # that protocol-level disconnects kill the socket so the
+      # recv loop unblocks with SocketDeadError.
+      def kill_on_protocol_error(event)
+        return unless event.type == :disconnected
+        error = event.detail && event.detail[:error]
+        return unless error.is_a?(Protocol::ZMTP::Error)
+        @sock.engine.signal_fatal_error(error)
       end
 
 
@@ -96,7 +102,7 @@ module OMQ
       def compile_expr
         @begin_proc, @end_proc, @eval_proc =
           OMQ::CLI::ExpressionEvaluator.compile_inside_ractor(@config.recv_expr)
-        @fmt = OMQ::CLI::Formatter.new(@config.format, compress: @config.compress)
+        @fmt = OMQ::CLI::Formatter.new(@config.format)
         @ctx = Object.new
         @ctx.instance_exec(&@begin_proc) if @begin_proc
       end
@@ -119,10 +125,8 @@ module OMQ
         n = @config.count
         i = 0
         loop do
-          wire = @sock.receive
-          break if wire.nil?
-          parts = @fmt.decompress(wire)
-          trace_in(parts, wire)
+          parts = @sock.receive
+          break if parts.nil?
           if @eval_proc
             parts = normalize(
               @ctx.instance_exec(parts, &@eval_proc)
@@ -145,15 +149,11 @@ module OMQ
         n = @config.count
         i = 0
         loop do
-          wire = @sock.receive
-          break if wire.nil?
-          parts = @fmt.decompress(wire)
-          trace_in(parts, wire)
+          parts = @sock.receive
+          break if parts.nil?
           reply = compute_reply(parts)
           output(reply)
-          reply_wire = @fmt.compress(reply)
-          trace_out(reply, reply_wire)
-          @sock.send(reply_wire)
+          @sock.send(reply)
           i += 1
           break if n && n > 0 && i >= n
         end

data/lib/omq/cli/pipe.rb

@@ -11,9 +11,8 @@ module OMQ
 
       # @param config [Config] frozen CLI configuration
       def initialize(config)
-        @config  = config
-        @fmt_in  = Formatter.new(config.format, compress: config.compress_in || config.compress)
-        @fmt_out = Formatter.new(config.format, compress: config.compress_out || config.compress)
+        @config = config
+        @fmt    = Formatter.new(config.format)
       end
 
 
@@ -51,12 +50,16 @@ module OMQ
         @pull, @push = build_pull_push(in_eps, out_eps)
         compile_expr
         @sock = @pull  # for eval instance_exec
-        start_event_monitors if config.verbose >= 2
+        start_event_monitors
         wait_for_peers_with_timeout if config.timeout
         setup_sequential_transient(task)
         @sock.instance_exec(&@recv_begin_proc) if @recv_begin_proc
         sequential_message_loop(fan_out: out_eps.size > 1)
         @sock.instance_exec(&@recv_end_proc) if @recv_end_proc
+      rescue OMQ::SocketDeadError => error
+        reason = error.cause&.message || error.message
+        $stderr.write("omq: #{reason}\n")
+        exit 1
       ensure
         @pull&.close
         @push&.close
@@ -85,6 +88,9 @@ module OMQ
         push = OMQ::PUSH.new(**kwargs)
         SocketSetup.apply_options(pull, config)
         SocketSetup.apply_options(push, config)
+        SocketSetup.apply_recv_maxsz(pull, config)
+        SocketSetup.apply_compression(pull, config, "pull")
+        SocketSetup.apply_compression(push, config, "push")
         SocketSetup.attach_endpoints(pull, in_eps, verbose: config.verbose, timestamps: config.timestamps)
         SocketSetup.attach_endpoints(push, out_eps, verbose: config.verbose, timestamps: config.timestamps)
         [pull, push]
@@ -107,10 +113,9 @@ module OMQ
         loop do
           parts = @pull.receive
           break if parts.nil?
-          parts = @fmt_in.decompress(parts)
           parts = eval_recv_expr(parts)
           if parts && !parts.empty?
-            @push.send(@fmt_out.compress(parts))
+            @push.send(parts)
           end
           # Yield after send so send-pump fibers can drain the queue
           # before the next message is enqueued. Without this, one pump
@@ -163,7 +168,7 @@ module OMQ
       def set_pipe_process_title
         in_eps, out_eps = resolve_endpoints
         title = ["omq pipe"]
-        title << "-z" if config.compress || config.compress_in || config.compress_out
+        title << "-z" if config.compress
         title << "-P#{config.parallel}" if config.parallel
         title.concat(in_eps.map(&:url))
         title << "->"
@@ -193,13 +198,23 @@ module OMQ
 
 
       def start_event_monitors
-        trace = config.verbose >= 3
+        trace      = config.verbose >= 3
+        log_events = config.verbose >= 2
         [@pull, @push].each do |sock|
           sock.monitor(verbose: trace) do |event|
-            Term.write_event(event, config.timestamps)
+            Term.write_event(event, config.timestamps) if log_events
+            kill_on_protocol_error(sock, event)
           end
         end
       end
+
+
+      def kill_on_protocol_error(sock, event)
+        return unless event.type == :disconnected
+        error = event.detail && event.detail[:error]
+        return unless error.is_a?(Protocol::ZMTP::Error)
+        sock.engine.signal_fatal_error(error)
+      end
     end
   end
 end

data/lib/omq/cli/pipe_worker.rb

@@ -19,13 +19,14 @@ module OMQ
         Async do
           setup_sockets
           log_endpoints if @config.verbose >= 1
-          start_monitors if @config.verbose >= 2
+          start_monitors
           wait_for_peers_with_timeout if @config.timeout
           compile_expr
           run_message_loop
           run_end_block
-        rescue OMQ::CLI::DecompressError => e
-          @error_port&.send(e.message)
+        rescue OMQ::SocketDeadError => error
+          reason = error.cause&.message || error.message
+          @log_port.send("omq: #{reason}")
         ensure
           @pull&.close
           @push&.close
@@ -42,6 +43,9 @@ module OMQ
         @push = OMQ::PUSH.new(**kwargs)
         OMQ::CLI::SocketSetup.apply_options(@pull, @config)
         OMQ::CLI::SocketSetup.apply_options(@push, @config)
+        OMQ::CLI::SocketSetup.apply_recv_maxsz(@pull, @config)
+        OMQ::CLI::SocketSetup.apply_compression(@pull, @config, "pull")
+        OMQ::CLI::SocketSetup.apply_compression(@push, @config, "push")
         OMQ::CLI::SocketSetup.attach_endpoints(@pull, @in_eps, verbose: 0)
         OMQ::CLI::SocketSetup.attach_endpoints(@push, @out_eps, verbose: 0)
       end
@@ -55,28 +59,22 @@ module OMQ
 
 
       def start_monitors
-        # When compress is on, messages are traced explicitly in
-        # run_message_loop post-decompression; keep engine-level
-        # traces off to avoid showing compressed bytes.
-        compress = @config.compress || @config.compress_in || @config.compress_out
-        trace    = @config.verbose >= 3 && !compress
+        trace      = @config.verbose >= 3
+        log_events = @config.verbose >= 2
         [@pull, @push].each do |sock|
           sock.monitor(verbose: trace) do |event|
-            @log_port.send(OMQ::CLI::Term.format_event(event, @config.timestamps))
+            @log_port.send(OMQ::CLI::Term.format_event(event, @config.timestamps)) if log_events
+            kill_on_protocol_error(sock, event)
           end
         end
       end
 
 
-      def trace_in(parts, wire)
-        return unless @config.verbose >= 3 && (@config.compress || @config.compress_in)
-        @log_port.send("#{OMQ::CLI::Term.log_prefix(@config.timestamps)}omq: << #{OMQ::CLI::Formatter.preview(parts, wire_size: wire.sum(&:bytesize))}")
-      end
-
-
-      def trace_out(parts, wire)
-        return unless @config.verbose >= 3 && (@config.compress || @config.compress_out)
-        @log_port.send("#{OMQ::CLI::Term.log_prefix(@config.timestamps)}omq: >> #{OMQ::CLI::Formatter.preview(parts, wire_size: wire.sum(&:bytesize))}")
+      def kill_on_protocol_error(sock, event)
+        return unless event.type == :disconnected
+        error = event.detail && event.detail[:error]
+        return unless error.is_a?(Protocol::ZMTP::Error)
+        sock.engine.signal_fatal_error(error)
       end
 
 
@@ -96,8 +94,7 @@ module OMQ
       def compile_expr
         @begin_proc, @end_proc, @eval_proc =
           OMQ::CLI::ExpressionEvaluator.compile_inside_ractor(@config.recv_expr)
-        @fmt_in  = OMQ::CLI::Formatter.new(@config.format, compress: @config.compress_in || @config.compress)
-        @fmt_out = OMQ::CLI::Formatter.new(@config.format, compress: @config.compress_out || @config.compress)
+        @fmt = OMQ::CLI::Formatter.new(@config.format)
         @ctx = Object.new
         @ctx.instance_exec(&@begin_proc) if @begin_proc
       end
@@ -124,29 +121,21 @@ module OMQ
 
 
       def process_one_eval
-        wire_in = @pull.receive
-        return false if wire_in.nil?
-        parts_in = @fmt_in.decompress(wire_in)
-        trace_in(parts_in, wire_in)
+        parts_in = @pull.receive
+        return false if parts_in.nil?
         parts_out = OMQ::CLI::ExpressionEvaluator.normalize_result(
           @ctx.instance_exec(parts_in, &@eval_proc), format: @config.format
         )
         return true if parts_out.nil? || parts_out.empty?
-        wire_out = @fmt_out.compress(parts_out)
-        trace_out(parts_out, wire_out)
-        @push << wire_out
+        @push << parts_out
         true
       end
 
 
       def process_one_passthrough
-        wire_in = @pull.receive
-        return false if wire_in.nil?
-        parts_in = @fmt_in.decompress(wire_in)
-        trace_in(parts_in, wire_in)
-        wire_out = @fmt_out.compress(parts_in)
-        trace_out(parts_in, wire_out)
-        @push << wire_out
+        parts_in = @pull.receive
+        return false if parts_in.nil?
+        @push << parts_in
         true
       end
 
@@ -156,7 +145,7 @@ module OMQ
         out = OMQ::CLI::ExpressionEvaluator.normalize_result(
           @ctx.instance_exec(&@end_proc), format: @config.format
         )
-        @push << @fmt_out.compress(out) if out && !out.empty?
+        @push << out if out && !out.empty?
       end
     end
   end

data/lib/omq/cli/radio_dish.rb

@@ -13,7 +13,6 @@ module OMQ
       def send_msg(parts)
         return if parts.empty?
         parts = [Marshal.dump(parts)] if config.format == :marshal
-        parts = @fmt.compress(parts)
         group = config.group || parts.shift
         @sock.publish(group, parts.first || "")
         transient_ready!

data/lib/omq/cli/router_dealer.rb

@@ -25,7 +25,6 @@ module OMQ
             break if parts.nil?
             identity = parts.shift
             parts.shift if parts.first == ""
-            parts = @fmt.decompress(parts)
             result = eval_recv_expr([display_routing_id(identity), *parts])
             output(result)
             i += 1
@@ -36,7 +35,7 @@ module OMQ
 
 
       def send_to_peer(id, parts)
-        @sock.send([id, "", *@fmt.compress(parts)])
+        @sock.send([id, "", *parts])
       end
     end
   end

data/lib/omq/cli/socket_setup.rb

@@ -22,6 +22,40 @@ module OMQ
       # it entirely with --recv-maxsz 0.
       DEFAULT_RECV_MAXSZ = 1 << 20
 
+      # Socket types that never receive application data. These opt
+      # out of the default passive-compression behavior -- they would
+      # otherwise advertise a profile and wrap every outgoing frame in
+      # a 4-byte uncompressed sentinel for no benefit, since there's
+      # nothing to decompress in return.
+      PURE_SEND_TYPES = %w[push pub scatter radio].freeze
+
+
+      # Installs ZMTP-Zstd compression on +sock+ based on +type_name+
+      # and the explicit flags in +config+. Three outcomes:
+      #
+      # * +config.compress+ is true --> active compression (auto-dict).
+      #   Outgoing frames are compressed; incoming are decoded.
+      # * +config.compress+ is false and +type_name+ can receive -->
+      #   passive compression (RFC Sec. 6.4). The socket advertises the
+      #   profile so an active peer can compress on the wire and we can
+      #   decode it, but we never compress our own outgoing frames.
+      # * +config.compress+ is false and +type_name+ is in
+      #   +PURE_SEND_TYPES+ --> no compression. Pure senders have no
+      #   incoming traffic to decompress, so passive mode is pure
+      #   overhead on outgoing.
+      #
+      # Callers pass +type_name+ explicitly (rather than reading it off
+      # +config+) so the pipe runners can install different modes on
+      # their push/pull ends of a single pipe.
+      def self.apply_compression(sock, config, type_name)
+        if config.compress
+          sock.compression = OMQ::RFC::Zstd::Compression.auto(level: config.compress_level || -3)
+        elsif !PURE_SEND_TYPES.include?(type_name)
+          sock.compression = OMQ::RFC::Zstd::Compression.auto(passive: true)
+        end
+      end
+
+
       # Apply common socket options from +config+ to +sock+.
       #
       def self.apply_options(sock, config)
@@ -44,16 +78,22 @@ module OMQ
         sock = config.ffi ? klass.new(backend: :ffi) : klass.new
         sock.conflate = true if config.conflate && %w[pub radio].include?(config.type_name)
         apply_options(sock, config)
-        # --recv-maxsz: nil → 1 MiB default; 0 → explicitly unlimited; else → as-is.
+        apply_recv_maxsz(sock, config)
+        sock.identity         = config.identity   if config.identity
+        sock.router_mandatory = true if config.type_name == "router"
+        apply_compression(sock, config, config.type_name)
+        sock
+      end
+
+
+      # --recv-maxsz: nil → 1 MiB default; 0 → explicitly unlimited; else → as-is.
+      def self.apply_recv_maxsz(sock, config)
         sock.max_message_size =
           case config.recv_maxsz
           when nil then DEFAULT_RECV_MAXSZ
           when 0   then nil
           else          config.recv_maxsz
           end
-        sock.identity         = config.identity   if config.identity
-        sock.router_mandatory = true if config.type_name == "router"
-        sock
       end
 
 

data/lib/omq/cli/term.rb

@@ -40,12 +40,23 @@ module OMQ
         prefix = log_prefix(timestamps)
         case event.type
         when :message_sent
-          "#{prefix}omq: >> #{Formatter.preview(event.detail[:parts])}"
+          "#{prefix}omq: >> #{Formatter.preview(event.detail[:parts], wire_size: event.detail[:wire_size])}"
         when :message_received
-          "#{prefix}omq: << #{Formatter.preview(event.detail[:parts])}"
+          "#{prefix}omq: << #{Formatter.preview(event.detail[:parts], wire_size: event.detail[:wire_size])}"
+        when :zdict_sent
+          "#{prefix}omq: >> ZDICT (#{event.detail[:size]}B)"
+        when :zdict_received
+          "#{prefix}omq: << ZDICT (#{event.detail[:size]}B)"
         else
           ep     = event.endpoint ? " #{event.endpoint}" : ""
-          detail = event.detail ? " #{event.detail}" : ""
+          detail =
+            if event.detail.is_a?(Hash) && event.detail[:reason]
+              " (#{event.detail[:reason]})"
+            elsif event.detail
+              " #{event.detail}"
+            else
+              ""
+            end
           "#{prefix}omq: #{event.type}#{ep}#{detail}"
         end
       end

data/lib/omq/cli/version.rb

@@ -2,6 +2,6 @@
 
 module OMQ
   module CLI
-    VERSION = "0.13.0"
+    VERSION = "0.14.0"
   end
 end

data/lib/omq/cli.rb

@@ -197,6 +197,7 @@ module OMQ
       require "omq/rfc/scattergather"
       require "omq/rfc/channel"
       require "omq/rfc/p2p"
+      require "omq/rfc/zstd"
       require "async"
       require "json"
       require "console"
@@ -247,9 +248,6 @@ module OMQ
                    runner_class.new(config)
                  end
         runner.call(task)
-      rescue DecompressError => e
-        $stderr.puts "omq: #{e.message}"
-        exit 1
       rescue IO::TimeoutError, Async::TimeoutError
         $stderr.puts "omq: timeout" unless config.quiet
         exit 2

metadata

@@ -1,7 +1,7 @@
 --- !ruby/object:Gem::Specification
 name: omq-cli
 version: !ruby/object:Gem::Version
-  version: 0.13.0
+  version: 0.14.0
 platform: ruby
 authors:
 - Patrik Wenger
@@ -15,14 +15,20 @@ dependencies:
     requirements:
     - - "~>"
       - !ruby/object:Gem::Version
-        version: '0.18'
+        version: '0.19'
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: 0.19.2
   type: :runtime
   prerelease: false
   version_requirements: !ruby/object:Gem::Requirement
     requirements:
     - - "~>"
       - !ruby/object:Gem::Version
-        version: '0.18'
+        version: '0.19'
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: 0.19.2
 - !ruby/object:Gem::Dependency
   name: omq-ffi
   requirement: !ruby/object:Gem::Requirement
@@ -108,51 +114,51 @@ dependencies:
       - !ruby/object:Gem::Version
         version: '0.1'
 - !ruby/object:Gem::Dependency
-  name: msgpack
+  name: omq-rfc-zstd
   requirement: !ruby/object:Gem::Requirement
     requirements:
-    - - ">="
+    - - "~>"
       - !ruby/object:Gem::Version
-        version: '0'
+        version: '0.1'
   type: :runtime
   prerelease: false
   version_requirements: !ruby/object:Gem::Requirement
     requirements:
-    - - ">="
+    - - "~>"
       - !ruby/object:Gem::Version
-        version: '0'
+        version: '0.1'
 - !ruby/object:Gem::Dependency
-  name: rbnacl
+  name: msgpack
   requirement: !ruby/object:Gem::Requirement
     requirements:
-    - - "~>"
+    - - ">="
       - !ruby/object:Gem::Version
-        version: '7.0'
+        version: '0'
   type: :runtime
   prerelease: false
   version_requirements: !ruby/object:Gem::Requirement
     requirements:
-    - - "~>"
+    - - ">="
       - !ruby/object:Gem::Version
-        version: '7.0'
+        version: '0'
 - !ruby/object:Gem::Dependency
-  name: rlz4
+  name: rbnacl
   requirement: !ruby/object:Gem::Requirement
     requirements:
     - - "~>"
       - !ruby/object:Gem::Version
-        version: '0.1'
+        version: '7.0'
   type: :runtime
   prerelease: false
   version_requirements: !ruby/object:Gem::Requirement
     requirements:
     - - "~>"
       - !ruby/object:Gem::Version
-        version: '0.1'
+        version: '7.0'
 description: Command-line tool for sending and receiving ZeroMQ messages on any socket
   type (REQ/REP, PUB/SUB, PUSH/PULL, DEALER/ROUTER, and all draft types). Supports
   Ruby eval (-e/-E), script handlers (-r), pipe virtual socket with Ractor parallelism,
-  multiple formats (ASCII, JSON Lines, msgpack, Marshal), LZ4 compression, and CURVE
+  multiple formats (ASCII, JSON Lines, msgpack, Marshal), Zstd compression, and CURVE
   encryption. Like nngcat from libnng, but with Ruby superpowers.
 email:
 - paddor@gmail.com