nnq-cli 0.2.0 → 0.4.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 2aac6e193d793225be560b51a32d4f3e713afbab8b25652be6cd76750f789e3e
-  data.tar.gz: a31568088dbc3ba5ad3df07f3e1f0f98543c1ad0c631718ae71f76d02b602c0a
+  metadata.gz: cfc34e5b7774c4c3288d440d5172391b098055fd2c376d3827e8a5944dd085d7
+  data.tar.gz: '0836095278d158e9b52234060cbfe111abcdc282a31659480ff2eb9db0bad5e8'
 SHA512:
-  metadata.gz: fe5db04b1ebb441274c4b182d3eabed59229c040593fe7f0388c123972a6059572c6a2dff3633e1f72abd0a6771b3b6e2812e6b00d07b9c4d75ad9bd831b8059
-  data.tar.gz: bba50ee147da4897cefce7297834584127183a04a6bb3a8c15525dabff70c04f865eaa8204dc69eebc991f72fc1cc1c01928ad5490be1ee164f656f39c70cc45
+  metadata.gz: d34031ee7de234d6f1f88305b8cbd7699932216bec86bb6f05191e6001608e899ff29160934cc9c9a34b9c8fc71c7bff2d74014904a85753bd63d947bf88f78b
+  data.tar.gz: 84a46b47822df0538eb9a875142b01893370c3222b22728ede2dfb1578354cd21775f22bc343daaded4bc60ba56f99af6a44dd0edb40f4181cab44040cc45fc7

data/CHANGELOG.md

@@ -1,5 +1,76 @@
 # Changelog
 
+## 0.4.0 — 2026-04-16
+
+- **`-P0` auto-sizes to `nproc`** (capped at 16). `nnq pipe -P 0`
+  now spawns one Ractor worker per online processor instead of
+  erroring. Explicit values still clamp to `1..16`.
+
+## 0.3.2 — 2026-04-16
+
+- **REQ generator mode (`-E`/`-e` with no stdin).** `nnq req` now
+  produces each request from the send-eval alone when no `-d`/`-F`
+  is given and stdin is not piped, matching the existing PUSH/PUB
+  generator behaviour. Bounded by `-n` or paced by `-i` like the
+  other generator-capable runners.
+- **System tests split into themed files under `test/system/`.**
+  Replaces the monolithic `test/system_test.sh` with 10 standalone
+  files sharing `test/system/support.sh` helpers.
+  `test/system/run_all.sh` chains them; each file also runs
+  standalone. New `rake test:system` task invokes `run_all.sh`.
+- **System tests for REQ/PUSH/PUB `-E` generator mode.** REQ fires
+  `-E'"foo"' -n 3` against a REP running `-e 'it.upcase' -n 3` and
+  verifies three `FOO` replies round-trip; PUSH and PUB get the
+  same treatment.
+
+## 0.3.1 — 2026-04-15
+
+- **Messages are single `String`s, not 1-element arrays.** Every
+  runner, the formatter, and the expression evaluator used to wrap
+  each message body in a one-element array as a historical
+  artifact of multipart thinking. That's gone now: `Formatter#pack`/
+  `#unpack` take and return a `String`, `Formatter.preview` takes a
+  `String`, and REQ/REP/SURVEYOR/RESPONDENT/pipe runners pass the
+  body straight through without `.first` or array literals. Fixes
+  an `undefined method 'first' for an instance of String` crash in
+  `ReqRunner#request_and_receive` that was masked by the tests.
+- **`-e` / `-E` expressions use Ruby 3.4's `it` default block
+  variable** (and accept explicit `|msg|` block-parameter syntax
+  via `proc { |msg| ... }`). The old implicit `$_`/`$F` aliasing is
+  gone; expressions are compiled as plain `proc { EXPR }` and
+  evaluated via `instance_exec(msg, &block)`, so `it.upcase` and
+  `|msg| msg.upcase` both work. Cleaner, faster, and no global
+  state.
+- **Requires nnq >= 0.6.1** for the `-vvv` trace fixes — cooked
+  REQ/REP/RESPONDENT now emit `>>` lines and recv previews strip
+  the SP backtrace header so you see the actual payload.
+- **`test/system_test.sh`** — new shell-based system test suite
+  modeled on omq-cli's, covering REQ/REP (basic, `--echo`, `-e`),
+  the ported `-vvv` REQ/REP verbose-trace assertion from omq-cli
+  commit `950890911de078`, PUSH/PULL, PUB/SUB, and the `@name`
+  abstract-namespace shortcut for both `-b` and `-c`.
+
+## 0.3.0 — 2026-04-15
+
+- **Compression switched from LZ4 to Zstd** via the new
+  [nnq-zstd](https://github.com/paddor/nnq-zstd) gem. `-z`
+  (fast, level −3) and `-Z` (balanced, level 3) are mutually
+  exclusive and map to transparent `NNQ::Zstd.wrap` decorators
+  around each socket. Sender-side dictionary training and in-band
+  dict shipping mean compression ratios on streams of similar
+  small messages are now dramatically better than the old
+  stateless LZ4 path. Both peers still have to pass `-z`/`-Z`;
+  there is no negotiation.
+- **`-Z` / `--compress-high`** flag for balanced Zstd.
+- **Formatter no longer knows about compression.** All per-message
+  compression state and logic moved out of `Formatter`; compression
+  is applied transparently at the socket boundary via the new
+  wrapper. Fewer code paths in runners.
+- **`rlz4` dependency dropped** in favor of `nnq-zstd ~> 0.1`.
+- **Lazy loading.** `nnq/zstd` (and therefore `rzstd` and the Rust
+  extension) is only required when `-z`/`-Z` is actually used,
+  keeping startup cost unchanged for non-compressing runs.
+
 ## 0.2.0 — 2026-04-15
 
 - **Peer wait for bind-mode bounded senders** — `nnq push -b ... -d

data/lib/nnq/cli/base_runner.rb

@@ -5,14 +5,15 @@ module NNQ
     # Template runner base class for all socket-type CLI runners.
     # Subclasses override {#run_loop} to implement socket-specific behaviour.
     #
-    # nnq carries one String body per message (no multipart). The runner
-    # protocol wraps it in a 1-element Array internally so that eval
-    # expressions using `$F`/`$_` work the same way as in omq-cli, and
-    # unwraps to a bare String at the send boundary.
+    # nnq carries one String body per message (no multipart).
+    #
     class BaseRunner
       # @return [Config] frozen CLI configuration
+      attr_reader :config
+
+
       # @return [Object] the NNQ socket instance
-      attr_reader :config, :sock
+      attr_reader :sock
 
 
       # @param config [Config] frozen CLI configuration
@@ -20,7 +21,7 @@ module NNQ
       def initialize(config, socket_class)
         @config = config
         @klass  = socket_class
-        @fmt    = Formatter.new(config.format, compress: config.compress)
+        @fmt    = Formatter.new(config.format)
       end
 
 
@@ -64,7 +65,8 @@ module NNQ
 
 
       def create_socket
-        SocketSetup.build(@klass, config)
+        sock = SocketSetup.build(@klass, config)
+        SocketSetup.maybe_wrap_zstd(sock, config.compress)
       end
 
 
@@ -144,6 +146,7 @@ module NNQ
       # latecomers finish their handshake before we start sending.
       def apply_grace_period
         return unless config.binds.any? || config.connects.size > 1
+
         ri = @sock.options.reconnect_interval
         sleep(ri.is_a?(Range) ? ri.begin : ri)
       end
@@ -162,7 +165,9 @@ module NNQ
 
       def run_send_logic
         n = config.count
-        sleep(config.delay) if config.delay
+
+        sleep config.delay if config.delay
+
         if config.interval
           run_interval_send(n)
         elsif config.data || config.file
@@ -188,21 +193,30 @@ module NNQ
 
       def run_interval_send(n)
         i = send_tick
-        return if @send_tick_eof || (n && n > 0 && i >= n)
+
+        if @send_tick_eof || (n && n > 0 && i >= n)
+          return
+        end
+
         Async::Loop.quantized(interval: config.interval) do
           i += send_tick
-          break if @send_tick_eof || (n && n > 0 && i >= n)
+
+          if @send_tick_eof || (n && n > 0 && i >= n)
+            break
+          end
         end
       end
 
 
       def run_stdin_send(n)
         i = 0
+
         loop do
-          msg = read_next
-          break unless msg
+          msg = read_next or break
           msg = eval_send_expr(msg)
+
           send_msg(msg) if msg
+
           i += 1
           break if n && n > 0 && i >= n
         end
@@ -211,6 +225,7 @@ module NNQ
 
       def send_tick
         raw = read_next_or_nil
+
         if raw.nil?
           if @send_eval_proc && !@stdin_ready
             # Pure generator mode: no stdin, eval produces output from nothing.
@@ -218,9 +233,11 @@ module NNQ
             send_msg(msg) if msg
             return 1
           end
+
           @send_tick_eof = true
           return 0
         end
+
         msg = eval_send_expr(raw)
         send_msg(msg) if msg
         1
@@ -230,14 +247,16 @@ module NNQ
       def run_recv_logic
         n = config.count
         i = 0
+
         if config.interval
           run_interval_recv(n)
         else
           loop do
-            msg = recv_msg
-            break if msg.nil?
+            msg = recv_msg or break
             msg = eval_recv_expr(msg)
+
             output(msg)
+
             i += 1
             break if n && n > 0 && i >= n
           end
@@ -247,21 +266,28 @@ module NNQ
 
       def run_interval_recv(n)
         i = recv_tick
+
         return if i == 0
         return if n && n > 0 && i >= n
+
         Async::Loop.quantized(interval: config.interval) do
           i += recv_tick
-          break if @recv_tick_eof || (n && n > 0 && i >= n)
+
+          if @recv_tick_eof || (n && n > 0 && i >= n)
+            break
+          end
         end
       end
 
 
       def recv_tick
         msg = recv_msg
+
         if msg.nil?
           @recv_tick_eof = true
           return 0
         end
+
         msg = eval_recv_expr(msg)
         output(msg)
         1
@@ -285,22 +311,30 @@ module NNQ
       # -- Message I/O -------------------------------------------------
 
 
-      # msg: 1-element Array. nnq sockets take a bare String body.
+      # @param msg [String]
+      #
       def send_msg(msg)
-        return if msg.empty?
-        msg = [Marshal.dump(msg.first)] if config.format == :marshal
-        msg = @fmt.compress(msg)
-        @sock.send(msg.first)
+        body = msg
+
+        case config.format
+        when :marshal
+          body = Marshal.dump(msg)
+        end
+
+        @sock.send(body)
         transient_ready!
       end
 
 
-      # @return [Array<String>, nil] 1-element Array (body), or nil on close.
+      # @return [String, nil] message body, or nil on close.
       def recv_msg
-        raw = @sock.receive
-        return nil if raw.nil?
-        msg = @fmt.decompress([raw])
-        msg = [Marshal.load(msg.first)] if config.format == :marshal
+        msg = @sock.receive or return
+
+        case config.format
+        when :marshal
+          msg = Marshal.load msg
+        end
+
         transient_ready!
         msg
       end
@@ -329,7 +363,7 @@ module NNQ
           @fmt.decode_marshal($stdin)
         when :raw
           data = $stdin.read
-          data.nil? || data.empty? ? nil : [data]
+          data.nil? || data.empty? ? nil : data
         else
           line = $stdin.gets
           line.nil? ? nil : @fmt.decode(line)
@@ -360,6 +394,7 @@ module NNQ
 
       def output(msg)
         return if config.quiet || msg.nil?
+
         $stdout.write(@fmt.encode(msg))
         $stdout.flush
       end
@@ -415,11 +450,13 @@ module NNQ
       def set_process_title(endpoints: nil)
         eps = endpoints || config.endpoints
         title = ["nnq", config.type_name]
-        title << "-z" if config.compress
+        title << (config.compress == :balanced ? "-Z" : "-z") if config.compress
         title << "-P#{config.parallel}" if config.parallel
+
         eps.each do |ep|
           title << (ep.respond_to?(:url) ? ep.url : ep.to_s)
         end
+
         Process.setproctitle(title.join(" "))
       end
 
@@ -429,6 +466,7 @@ module NNQ
 
       def log(msg)
         return unless config.verbose >= 1
+
         $stderr.write("#{Term.log_prefix(config.verbose)}nnq: #{msg}\n")
       end
 
@@ -438,11 +476,13 @@ module NNQ
       # -vvvv: prepend ISO8601 timestamps
       def start_event_monitor
         verbose = config.verbose >= 3
-        v = config.verbose
+        v       = config.verbose
+
         @sock.monitor(verbose: verbose) do |event|
           CLI::Term.write_event(event, v)
         end
       end
+
     end
   end
 end

data/lib/nnq/cli/cli_parser.rb

@@ -10,13 +10,13 @@ module NNQ
       EXAMPLES = <<~'TEXT'
         -- Request / Reply ------------------------------------------
 
-          +-----+  "hello"    +-----+
+          +-----+   "hello"    +-----+
           | REQ |------------->| REP |
           |     |<-------------|     |
-          +-----+  "HELLO"    +-----+
+          +-----+   "HELLO"    +-----+
 
           # terminal 1: echo server
-          nnq rep --bind tcp://:5555 --recv-eval '$_.upcase'
+          nnq rep --bind tcp://:5555 --recv-eval 'it.upcase'
 
           # terminal 2: send a request
           echo "hello" | nnq req --connect tcp://localhost:5555
@@ -27,9 +27,9 @@ module NNQ
 
         -- Publish / Subscribe --------------------------------------
 
-          +-----+  "weather.nyc 72F"  +-----+
+          +-----+   "weather.nyc 72F"  +-----+
           | PUB |--------------------->| SUB | --subscribe "weather."
-          +-----+                     +-----+
+          +-----+                      +-----+
 
           # terminal 1: subscriber (all topics by default)
           nnq sub --bind tcp://:5556
@@ -39,9 +39,9 @@ module NNQ
 
         -- Periodic Publish -------------------------------------------
 
-          +-----+  "tick 1"    +-----+
+          +-----+   "tick 1"    +-----+
           | PUB |--(every 1s)-->| SUB |
-          +-----+              +-----+
+          +-----+               +-----+
 
           # terminal 1: subscriber
           nnq sub --bind tcp://:5556
@@ -56,9 +56,9 @@ module NNQ
 
         -- Pipeline -------------------------------------------------
 
-          +------+           +------+
+          +------+            +------+
           | PUSH |----------->| PULL |
-          +------+           +------+
+          +------+            +------+
 
           # terminal 1: worker
           nnq pull --bind tcp://:5557
@@ -72,30 +72,30 @@ module NNQ
 
         -- Pipe (PULL -> eval -> PUSH) --------------------------------
 
-          +------+         +------+         +------+
+          +------+          +------+          +------+
           | PUSH |--------->| pipe |--------->| PULL |
-          +------+         +------+         +------+
+          +------+          +------+          +------+
 
           # terminal 1: producer
           echo -e "hello\nworld" | nnq push --bind ipc://@work
 
           # terminal 2: worker -- uppercase each message
-          nnq pipe -c ipc://@work -c ipc://@sink -e '$_.upcase'
+          nnq pipe -c ipc://@work -c ipc://@sink -e 'it.upcase'
           # terminal 3: collector
           nnq pull --bind ipc://@sink
 
           # 4 Ractor workers in a single process (-P)
-          nnq pipe -c ipc://@work -c ipc://@sink -P4 -r./fib -e 'fib(Integer($_)).to_s'
+          nnq pipe -c ipc://@work -c ipc://@sink -P4 -r./fib -e 'fib(Integer(it)).to_s'
 
           # exit when producer disconnects (--transient)
-          nnq pipe -c ipc://@work -c ipc://@sink --transient -e '$_.upcase'
+          nnq pipe -c ipc://@work -c ipc://@sink --transient -e 'it.upcase'
 
           # fan-in: multiple sources -> one sink
           nnq pipe --in -c ipc://@work1 -c ipc://@work2 \
-            --out -c ipc://@sink -e '$_.upcase'
+            --out -c ipc://@sink -e 'it.upcase'
 
           # fan-out: one source -> multiple sinks (round-robin)
-          nnq pipe --in -b tcp://:5555 --out -c ipc://@sink1 -c ipc://@sink2 -e '$_'
+          nnq pipe --in -b tcp://:5555 --out -c ipc://@sink1 -c ipc://@sink2 -e 'it'
 
         -- Formats --------------------------------------------------
 
@@ -105,10 +105,6 @@ module NNQ
           # quoted -- lossless, round-trippable (uses String#dump escaping)
           nnq pull --bind tcp://:5557 --quoted
 
-          # JSON Lines -- single-element arrays (nnq is single-body)
-          echo '["payload"]' | nnq push --connect tcp://localhost:5557 --jsonl
-          nnq pull --bind tcp://:5557 --jsonl
-
           # raw -- emit the body verbatim (no framing, no newline)
           nnq pull --bind tcp://:5557 --raw
 
@@ -121,25 +117,25 @@ module NNQ
         -- Ruby Eval ------------------------------------------------
 
           # filter incoming: only pass messages containing "error"
-          nnq pull -b tcp://:5557 --recv-eval '$_.include?("error") ? $_ : nil'
+          nnq pull -b tcp://:5557 --recv-eval 'it.include?("error") ? it : nil'
 
           # transform incoming with gems
-          nnq sub -c tcp://localhost:5556 -rjson -e 'JSON.parse($_)["temperature"]'
+          nnq sub -c tcp://localhost:5556 -rjson -e 'JSON.parse(it)["temperature"]'
 
           # require a local file, use its methods
-          nnq rep --bind tcp://:5555 --require ./transform.rb -e 'upcase($_)'
+          nnq rep --bind tcp://:5555 --require ./transform.rb -e 'transform(it)'
 
-          # next skips, break stops -- regexps match against $_
-          nnq pull -b tcp://:5557 -e 'next if /^#/; break if /quit/; $_'
+          # next skips, break stops
+          nnq pull -b tcp://:5557 -e 'next if /^#/; break if it =~ /quit/; it'
 
           # BEGIN/END blocks (like awk) -- accumulate and summarize
-          nnq pull -b tcp://:5557 -e 'BEGIN{@sum = 0} @sum += Integer($_); nil END{puts @sum}'
+          nnq pull -b tcp://:5557 -e 'BEGIN{@sum = 0} @sum += Integer(it); nil END{puts @sum}'
 
-          # transform outgoing messages
-          echo hello | nnq push -c tcp://localhost:5557 --send-eval '$_.upcase'
+          # transform outgoing messages (with explicit block variable msg)
+          echo hello | nnq push -c tcp://localhost:5557 --send-eval '|msg| msg.upcase'
 
           # REQ: transform request and reply independently
-          echo hello | nnq req -c tcp://localhost:5555 -E '$_.upcase' -e '$_'
+          echo hello | nnq req -c tcp://localhost:5555 -E 'it.upcase' -e 'it'
 
         -- Script Handlers (-r) ------------------------------------
 
@@ -150,7 +146,7 @@ module NNQ
           nnq pull --bind tcp://:5557 -r./handler.rb
 
           # combine script handlers with inline eval
-          nnq req -c tcp://localhost:5555 -r./handler.rb -E '$_.upcase'
+          nnq req -c tcp://localhost:5555 -r./handler.rb -E 'it.upcase'
 
           # NNQ.outgoing { |msg| ... }   -- registered outgoing transform
           # NNQ.incoming { |msg| ... }   -- registered incoming transform
@@ -178,9 +174,9 @@ module NNQ
         send_hwm:      nil,
         sndbuf:        nil,
         rcvbuf:        nil,
-        compress:      false,
-        compress_in:   false,
-        compress_out:  false,
+        compress:      nil,
+        compress_in:   nil,
+        compress_out:  nil,
         send_expr:     nil,
         recv_expr:     nil,
         parallel:      nil,
@@ -268,7 +264,6 @@ module NNQ
           o.on("-A", "--ascii",   "Safe ASCII, non-printable as dots (default)") { opts[:format] = :ascii }
           o.on("-Q", "--quoted",  "C-style quoted with escapes")                 { opts[:format] = :quoted }
           o.on(      "--raw",     "Raw binary body, no framing, no newline")    { opts[:format] = :raw }
-          o.on("-J", "--jsonl",   "JSON Lines (single-element array per line)") { opts[:format] = :jsonl }
           o.on(      "--msgpack", "MessagePack (binary stream)")                 { require "msgpack"; opts[:format] = :msgpack }
           o.on("-M", "--marshal", "Ruby Marshal stream (binary)")                 { opts[:format] = :marshal }
 
@@ -295,27 +290,37 @@ module NNQ
           o.on("--rcvbuf N", "SO_RCVBUF kernel buffer size (e.g. 4K, 1M)") { |v| opts[:rcvbuf] = parse_byte_size(v) }
 
           o.separator "\nCompression:"
-          o.on("-z", "--compress", "LZ4 compression per message (modal with --in/--out)") do
-            require "rlz4"
-            case pipe_side
-            when :in
-              opts[:compress_in] = true
-            when :out
-              opts[:compress_out] = true
-            else
-              opts[:compress] = true
+          load_zstd = -> { require "nnq/zstd" }
+          set_compress = lambda do |sym|
+            load_zstd.call
+            target = case pipe_side
+                     when :in  then :compress_in
+                     when :out then :compress_out
+                     else           :compress
+                     end
+            if opts[target] && opts[target] != sym
+              abort "nnq: -z and -Z are mutually exclusive"
             end
+            opts[target] = sym
+          end
+          o.on("-z", "--compress", "Zstd compression (fast, level -3; modal with --in/--out)") do
+            set_compress.call(:fast)
+          end
+          o.on("-Z", "--compress-high", "Zstd compression (balanced, level 3; modal with --in/--out)") do
+            set_compress.call(:balanced)
           end
 
           o.separator "\nProcessing (-e = incoming, -E = outgoing):"
-          o.on("-e", "--recv-eval EXPR", "Eval Ruby for each incoming message ($_ = body)") { |v| opts[:recv_expr] = v }
-          o.on("-E", "--send-eval EXPR", "Eval Ruby for each outgoing message ($_ = body)") { |v| opts[:send_expr] = v }
+          o.on("-e", "--recv-eval EXPR", "Eval Ruby for each incoming message (it = msg)") { |v| opts[:recv_expr] = v }
+          o.on("-E", "--send-eval EXPR", "Eval Ruby for each outgoing message (it = msg)") { |v| opts[:send_expr] = v }
           o.on("-r", "--require LIB",  "Require lib/file in Async context; use '-' for stdin. Scripts can register NNQ.outgoing/incoming") { |v|
             require "nnq" unless defined?(NNQ::VERSION)
             opts[:scripts] << (v == "-" ? :stdin : (v.start_with?("./", "../") ? File.expand_path(v) : v))
           }
-          o.on("-P", "--parallel N", Integer, "Parallel Ractor workers (max 16)") { |v|
-            opts[:parallel] = [v, 16].min
+          o.on("-P", "--parallel N", Integer, "Parallel Ractor workers, 1..16 (0 = nproc, capped at 16)") { |v|
+            require "etc"
+            resolved = v.zero? ? Etc.nprocessors : v
+            opts[:parallel] = [resolved, 16].min
           }
 
           o.separator "\nOther:"

data/lib/nnq/cli/expression_evaluator.rb

@@ -8,8 +8,10 @@ module NNQ
     #
     # One instance per direction (send or recv).
     #
-    # nnq has no multipart, so `$F` is always a 1-element array and `$_`
-    # is the body string.
+    # The expression sees the message body via the default block
+    # variable `it`, or callers can declare an explicit parameter with
+    # block-literal syntax: `-e '|msg| msg.upcase'` compiles to
+    # `proc { |msg| msg.upcase }`.
     #
     class ExpressionEvaluator
       attr_reader :begin_proc, :end_proc, :eval_proc
@@ -28,34 +30,30 @@ module NNQ
 
         if src
           expr, begin_body, end_body = extract_blocks(src)
-          @begin_proc = eval("proc { #{begin_body} }") if begin_body # rubocop:disable Security/Eval
-          @end_proc   = eval("proc { #{end_body} }")   if end_body   # rubocop:disable Security/Eval
+          @begin_proc = eval("proc { #{begin_body} }") if begin_body
+          @end_proc   = eval("proc { #{end_body} }")   if end_body
+
           if expr && !expr.strip.empty?
-            @eval_proc = eval("proc { $_ = $F&.first; #{expr} }") # rubocop:disable Security/Eval
+            @eval_proc = eval("proc { #{expr} }")
           end
         elsif fallback_proc
-          @eval_proc = proc { |msg|
-            body = msg&.first
-            $_ = body
-            fallback_proc.call(body)
-          }
+          @eval_proc = proc { |msg| fallback_proc.call(msg) }
         end
       end
 
 
       # Runs the eval proc against +msg+ using +context+ as self.
-      # Returns the normalised result Array, nil (filter/skip), or SENT.
+      # Returns the normalised result (as String), nil (filter/skip), or SENT.
+      #
       def call(msg, context)
         return msg unless @eval_proc
 
-        $F     = msg
         result = context.instance_exec(msg, &@eval_proc)
         return nil  if result.nil?
         return SENT if result.equal?(context)
-        return [result] if @format == :marshal
+        return result if @format == :marshal
 
-        result = result.is_a?(Array) ? result.first(1) : [result]
-        result.map!(&:to_s)
+        result.to_s
       rescue => e
         $stderr.puts "nnq: eval error: #{e.message} (#{e.class})"
         exit 3
@@ -66,8 +64,7 @@ module NNQ
       # Used inside Ractor worker blocks.
       def self.normalize_result(result)
         return nil if result.nil?
-        result = result.is_a?(Array) ? result.first(1) : [result]
-        result.map!(&:to_s)
+        result.to_s
       end
 
 
@@ -77,52 +74,34 @@ module NNQ
       def self.compile_inside_ractor(src)
         return [nil, nil, nil] unless src
 
-        extract = ->(expr, kw) {
-          s = expr.index(/#{kw}\s*\{/)
-          return [expr, nil] unless s
-          ci = expr.index("{", s)
-          depth = 1
-          j = ci + 1
-          while j < expr.length && depth > 0
-            depth += 1 if expr[j] == "{"
-            depth -= 1 if expr[j] == "}"
-            j += 1
-          end
-          [expr[0...s] + expr[j..], expr[(ci + 1)..(j - 2)]]
-        }
-
-        expr, begin_body = extract.(src, "BEGIN")
-        expr, end_body   = extract.(expr, "END")
+        expr, begin_body = extract_block(src,  "BEGIN")
+        expr, end_body   = extract_block(expr, "END")
 
-        begin_proc = eval("proc { #{begin_body} }") if begin_body # rubocop:disable Security/Eval
-        end_proc   = eval("proc { #{end_body} }")   if end_body   # rubocop:disable Security/Eval
+        begin_proc = eval("proc { #{begin_body} }") if begin_body
+        end_proc   = eval("proc { #{end_body} }")   if end_body
         eval_proc  = nil
+
         if expr && !expr.strip.empty?
-          ractor_expr = expr.gsub(/\$F\b/, "__F")
-          eval_proc   = eval("proc { |__F| $_ = __F&.first; #{ractor_expr} }") # rubocop:disable Security/Eval
+          eval_proc = eval("proc { #{expr} }")
         end
 
         [begin_proc, end_proc, eval_proc]
       end
 
 
-      private
-
-
-      def extract_blocks(expr)
-        expr, begin_body = extract_block(expr, "BEGIN")
-        expr, end_body   = extract_block(expr, "END")
-        [expr, begin_body, end_body]
-      end
-
-
-      def extract_block(expr, keyword)
-        start = expr.index(/#{keyword}\s*\{/)
-        return [expr, nil] unless start
+      # Strips a +BEGIN {...}+ or +END {...}+ block from +expr+ and
+      # returns +[trimmed_expr, block_body_or_nil]+. Brace-matched scan,
+      # so nested `{}` inside the block body are handled. Shared by
+      # instance and Ractor compile paths, so must be a class method
+      # (Ractors cannot call back into instance state).
+      #
+      def self.extract_block(expr, keyword)
+        start = expr.index(/#{keyword}\s*\{/) or return [expr, nil]
 
         i     = expr.index("{", start)
         depth = 1
         j     = i + 1
+
         while j < expr.length && depth > 0
           case expr[j]
           when "{"
@@ -130,6 +109,7 @@ module NNQ
           when "}"
             depth -= 1
           end
+
           j += 1
         end
 
@@ -137,6 +117,17 @@ module NNQ
         trimmed = expr[0...start] + expr[j..]
         [trimmed, body]
       end
+
+
+      private
+
+
+      def extract_blocks(expr)
+        expr, begin_body = self.class.extract_block(expr, "BEGIN")
+        expr, end_body   = self.class.extract_block(expr, "END")
+        [expr, begin_body, end_body]
+      end
+
     end
   end
 end

data/lib/nnq/cli/formatter.rb

@@ -2,66 +2,54 @@
 
 module NNQ
   module CLI
-    # Raised when LZ4 decompression fails.
-    class DecompressError < RuntimeError; end
-
     # Handles encoding/decoding a single-body message in the configured
-    # format, plus optional LZ4 compression.
+    # format. Compression is handled by the NNQ::Zstd decorator around
+    # the socket, not by the formatter.
     #
     # Unlike omq-cli's Formatter, nnq messages are not multipart — one
-    # `String` body per message. The API still accepts/returns a
-    # 1-element array so that `$F`-based eval expressions work the same
-    # way.
+    # `String` body per message. The API takes and returns a plain
+    # `String`.
     class Formatter
-      # @param format [Symbol] wire format (:ascii, :quoted, :raw, :jsonl, :msgpack, :marshal)
-      # @param compress [Boolean] whether to apply LZ4 compression
-      def initialize(format, compress: false)
-        @format   = format
-        @compress = compress
+      # @param format [Symbol] wire format (:ascii, :quoted, :raw, :msgpack, :marshal)
+      def initialize(format)
+        @format = format
       end
 
 
       # Encodes a message body into a printable string for output.
       #
-      # @param msg [Array<String>] single-element array (the body)
+      # @param msg [String] message body
       # @return [String] formatted output line
+      #
       def encode(msg)
-        body = msg.first.to_s
         case @format
         when :ascii
-          body.b.gsub(/[^[:print:]\t]/, ".") + "\n"
+          msg.b.gsub(/[^[:print:]\t]/, ".") << "\n"
         when :quoted
-          body.b.dump[1..-2] + "\n"
+          msg.b.dump[1..-2] << "\n"
         when :raw
-          body
-        when :jsonl
-          JSON.generate([body]) + "\n"
+          msg # FIXME: are these really the wire bytes?
         when :msgpack
-          MessagePack.pack([body])
+          MessagePack.pack(msg)
         when :marshal
-          body.inspect + "\n"
+          msg.inspect << "\n"
         end
       end
 
 
-      # Decodes a formatted input line into a 1-element message array.
+      # Decodes a formatted input line into a message body.
       #
       # @param line [String] input line (newline-terminated)
-      # @return [Array<String>] 1-element array
+      # @return [String] message
+      #
       def decode(line)
         case @format
         when :ascii, :marshal
-          [line.chomp]
+          line.chomp
         when :quoted
-          ["\"#{line.chomp}\"".undump]
+          "\"#{line.chomp}\"".undump
         when :raw
-          [line]
-        when :jsonl
-          arr = JSON.parse(line.chomp)
-          unless arr.is_a?(Array) && arr.all? { |e| e.is_a?(String) }
-            abort "JSON Lines input must be an array of strings"
-          end
-          arr.first(1)
+          line
         end
       end
 
@@ -89,33 +77,12 @@ module NNQ
       end
 
 
-      # Compresses the body with LZ4 if compression is enabled.
-      #
-      # @param msg [Array<String>] single-element array
-      # @return [Array<String>] optionally compressed
-      def compress(msg)
-        @compress ? msg.map { |p| RLZ4.compress(p) if p } : msg
-      end
-
-
-      # Decompresses the body with LZ4 if compression is enabled.
-      # nil/empty bodies pass through.
-      #
-      # @param msg [Array<String>] possibly compressed single-element array
-      # @return [Array<String>] decompressed
-      def decompress(msg)
-        @compress ? msg.map { |p| p && !p.empty? ? RLZ4.decompress(p) : p } : msg
-      rescue RLZ4::DecompressError
-        raise DecompressError, "decompression failed (did the sender use --compress?)"
-      end
-
-
       # Formats a message body for human-readable preview (logging).
       #
-      # @param msg [Array<String>] single-element array
+      # @param body [String] message body
       # @return [String] truncated preview
-      def self.preview(msg)
-        body = msg.first.to_s
+      def self.preview(body)
+        body = body.to_s
         "(#{body.bytesize}B) #{preview_body(body)}"
       end
 

data/lib/nnq/cli/pipe.rb

@@ -12,8 +12,8 @@ module NNQ
       # @param config [Config] frozen CLI configuration
       def initialize(config)
         @config  = config
-        @fmt_in  = Formatter.new(config.format, compress: config.compress_in || config.compress)
-        @fmt_out = Formatter.new(config.format, compress: config.compress_out || config.compress)
+        @fmt_in  = Formatter.new(config.format)
+        @fmt_out = Formatter.new(config.format)
       end
 
 
@@ -68,6 +68,8 @@ module NNQ
         push = SocketSetup.build(NNQ::PUSH0, config)
         SocketSetup.attach_endpoints(pull, in_eps, verbose: config.verbose)
         SocketSetup.attach_endpoints(push, out_eps, verbose: config.verbose)
+        pull = SocketSetup.maybe_wrap_zstd(pull, config.compress_in || config.compress)
+        push = SocketSetup.maybe_wrap_zstd(push, config.compress_out || config.compress)
         [pull, push]
       end
 
@@ -103,13 +105,9 @@ module NNQ
         loop do
           body = @pull.receive
           break if body.nil?
-          msg = @fmt_in.decompress([body])
-          msg = eval_recv_expr(msg)
+          msg = eval_recv_expr(body)
 
-          if msg && !msg.empty?
-            out = @fmt_out.compress(msg)
-            @push.send(out.first)
-          end
+          @push.send(msg) if msg
 
           # Yield after send so send-pump fibers can drain the queue
           # before the next message is enqueued. Without this, one pump
@@ -163,7 +161,9 @@ module NNQ
       def set_pipe_process_title
         in_eps, out_eps = resolve_endpoints
         title = ["nnq pipe"]
-        title << "-z" if config.compress || config.compress_in || config.compress_out
+        if (m = config.compress || config.compress_in || config.compress_out)
+          title << (m == :balanced ? "-Z" : "-z")
+        end
         title << "-P#{config.parallel}" if config.parallel
         title.concat(in_eps.map(&:url))
         title << "->"

data/lib/nnq/cli/pipe_worker.rb

@@ -24,8 +24,8 @@ module NNQ
           compile_expr
           run_message_loop
           run_end_block
-        rescue NNQ::CLI::DecompressError => e
-          @error_port&.send(e.message)
+        rescue NNQ::Zstd::ProtocolError => e
+          @error_port&.send("zstd protocol error: #{e.message}")
         ensure
           @pull&.close
           @push&.close
@@ -41,6 +41,8 @@ module NNQ
         @push = NNQ::CLI::SocketSetup.build(NNQ::PUSH0, @config)
         NNQ::CLI::SocketSetup.attach_endpoints(@pull, @in_eps, verbose: 0)
         NNQ::CLI::SocketSetup.attach_endpoints(@push, @out_eps, verbose: 0)
+        @pull = NNQ::CLI::SocketSetup.maybe_wrap_zstd(@pull, @config.compress_in || @config.compress)
+        @push = NNQ::CLI::SocketSetup.maybe_wrap_zstd(@push, @config.compress_out || @config.compress)
       end
 
 
@@ -63,9 +65,9 @@ module NNQ
       def format_event(event)
         case event.type
         when :message_sent
-          "nnq: >> #{NNQ::CLI::Formatter.preview([event.detail[:body]])}"
+          "nnq: >> #{NNQ::CLI::Formatter.preview(event.detail[:body])}"
         when :message_received
-          "nnq: << #{NNQ::CLI::Formatter.preview([event.detail[:body]])}"
+          "nnq: << #{NNQ::CLI::Formatter.preview(event.detail[:body])}"
         else
           ep     = event.endpoint ? " #{event.endpoint}" : ""
           detail = event.detail ? " #{event.detail}" : ""
@@ -86,8 +88,6 @@ module NNQ
       def compile_expr
         @begin_proc, @end_proc, @eval_proc =
           NNQ::CLI::ExpressionEvaluator.compile_inside_ractor(@config.recv_expr)
-        @fmt_in  = NNQ::CLI::Formatter.new(@config.format, compress: @config.compress_in || @config.compress)
-        @fmt_out = NNQ::CLI::Formatter.new(@config.format, compress: @config.compress_out || @config.compress)
         @ctx = Object.new
         @ctx.instance_exec(&@begin_proc) if @begin_proc
       end
@@ -100,12 +100,9 @@ module NNQ
             body = @pull.receive
             break if body.nil?
             msg = NNQ::CLI::ExpressionEvaluator.normalize_result(
-              @ctx.instance_exec(@fmt_in.decompress([body]), &@eval_proc)
+              @ctx.instance_exec(body, &@eval_proc)
             )
-            unless msg.nil? || msg.empty?
-              out = @fmt_out.compress(msg)
-              @push.send(out.first)
-            end
+            @push.send(msg) if msg
             n -= 1 if n && n > 0
             break if n == 0
           end
@@ -113,8 +110,7 @@ module NNQ
           loop do
             body = @pull.receive
             break if body.nil?
-            out = @fmt_out.compress(@fmt_in.decompress([body]))
-            @push.send(out.first)
+            @push.send(body)
             n -= 1 if n && n > 0
             break if n == 0
           end
@@ -129,9 +125,7 @@ module NNQ
         out = NNQ::CLI::ExpressionEvaluator.normalize_result(
           @ctx.instance_exec(&@end_proc)
         )
-        if out && !out.empty?
-          @push.send(@fmt_out.compress(out).first)
-        end
+        @push.send(out) if out
       end
     end
   end

data/lib/nnq/cli/req_rep.rb

@@ -13,33 +13,39 @@ module NNQ
       def run_loop(task)
         n = config.count
         i = 0
-        sleep(config.delay) if config.delay
+
+        sleep config.delay if config.delay
+        generator = @send_eval_proc && !config.data && !config.file && !stdin_ready?
+
         loop do
-          msg = read_next
-          break unless msg
-          msg = eval_send_expr(msg)
+          if generator
+            msg = eval_send_expr(nil)
+          else
+            msg = read_next
+            break unless msg
+            msg = eval_send_expr(msg)
+          end
+
           next unless msg
+
           reply = request_and_receive(msg)
           break if reply.nil?
           output(eval_recv_expr(reply))
           i += 1
           break if n && n > 0 && i >= n
-          break if !config.interval && (config.data || config.file)
+          break if !config.interval && (generator || config.data || config.file) && !(n && n > 0)
           wait_for_interval if config.interval
         end
       end
 
 
       def request_and_receive(msg)
-        return nil if msg.empty?
-        msg = [Marshal.dump(msg.first)] if config.format == :marshal
-        msg = @fmt.compress(msg)
-        reply_body = @sock.send_request(msg.first)
+        return nil if msg.nil? || msg.empty?
+        body = config.format == :marshal ? Marshal.dump(msg) : msg
+        reply_body = @sock.send_request(body)
         transient_ready!
         return nil if reply_body.nil?
-        reply = @fmt.decompress([reply_body])
-        reply = [Marshal.load(reply.first)] if config.format == :marshal
-        reply
+        config.format == :marshal ? Marshal.load(reply_body) : reply_body
       end
 
 
@@ -76,7 +82,7 @@ module NNQ
           reply = eval_recv_expr(msg)
           unless reply.equal?(SENT)
             output(reply)
-            send_reply(reply || [""])
+            send_reply(reply || "")
           end
         elsif config.echo
           output(msg)
@@ -94,10 +100,9 @@ module NNQ
 
 
       def send_reply(msg)
-        return if msg.empty?
-        msg = [Marshal.dump(msg.first)] if config.format == :marshal
-        msg = @fmt.compress(msg)
-        @sock.send_reply(msg.first)
+        return if msg.nil?
+        body = config.format == :marshal ? Marshal.dump(msg) : msg
+        @sock.send_reply(body)
         transient_ready!
       end
     end

data/lib/nnq/cli/socket_setup.rb

@@ -10,12 +10,14 @@ module NNQ
       # (one batch exactly fills a full queue).
       DEFAULT_HWM = 64
 
+
       # Default max inbound message size (1 MiB) so a misconfigured or
       # malicious peer can't force arbitrary memory allocation on a
       # terminal user. Users can raise it with --recv-maxsz N, or
       # disable it entirely with --recv-maxsz 0.
       DEFAULT_RECV_MAXSZ = 1 << 20
 
+
       # Apply post-construction socket options from +config+ to +sock+.
       # send_hwm and linger are construction-time kwargs (see {.build});
       # the rest of the options are set here and read later by the
@@ -24,12 +26,15 @@ module NNQ
         sock.options.read_timeout       = config.timeout       if config.timeout
         sock.options.write_timeout      = config.timeout       if config.timeout
         sock.options.reconnect_interval = config.reconnect_ivl if config.reconnect_ivl
-        sock.options.max_message_size =
-          case config.recv_maxsz
-          when nil then DEFAULT_RECV_MAXSZ
-          when 0   then nil
-          else          config.recv_maxsz
-          end
+
+        case config.recv_maxsz
+        when nil
+          sock.options.max_message_size = DEFAULT_RECV_MAXSZ
+        when 0  
+          sock.options.max_message_size = nil
+        else
+          config.recv_maxsz
+        end
       end
 
 
@@ -38,10 +43,10 @@ module NNQ
       # (send_hwm is captured during routing init and can't be changed
       # after the fact), so we pass them there.
       def self.build(klass, config)
-        sock = klass.new(
-          linger:   config.linger,
-          send_hwm: config.send_hwm || DEFAULT_HWM,
-        )
+        linger   = config.linger
+        send_hwm = config.send_hwm || DEFAULT_HWM
+        sock     = klass.new(linger:, send_hwm:)
+
         apply_options(sock, config)
         sock
       end
@@ -54,6 +59,7 @@ module NNQ
           sock.bind(url)
           CLI::Term.write_attach(:bind, sock.last_endpoint, verbose) if verbose >= 1
         end
+
         config.connects.each do |url|
           sock.connect(url)
           CLI::Term.write_attach(:connect, url, verbose) if verbose >= 1
@@ -77,6 +83,16 @@ module NNQ
       end
 
 
+      # Wrap +sock+ with NNQ::Zstd if +mode+ is :fast or :balanced.
+      # Returns the wrapper (decorator) or +sock+ unchanged.
+      def self.maybe_wrap_zstd(sock, mode)
+        return sock unless mode
+        require "nnq/zstd"
+        level = mode == :balanced ? 3 : -3
+        NNQ::Zstd.wrap(sock, level: level)
+      end
+
+
       # Subscribe to prefixes on a SUB socket.
       #
       # Unlike ZeroMQ, nng's sub0 starts with an empty subscription set,
@@ -88,6 +104,7 @@ module NNQ
         prefixes = config.subscribes.empty? ? [""] : config.subscribes
         prefixes.each { |p| sock.subscribe(p) }
       end
+
     end
   end
 end

data/lib/nnq/cli/surveyor_respondent.rb

@@ -29,10 +29,9 @@ module NNQ
 
 
       def survey_and_collect(msg)
-        return if msg.empty?
-        msg = [Marshal.dump(msg.first)] if config.format == :marshal
-        msg = @fmt.compress(msg)
-        @sock.send_survey(msg.first)
+        return if msg.nil? || msg.empty?
+        body = config.format == :marshal ? Marshal.dump(msg) : msg
+        @sock.send_survey(body)
         transient_ready!
         collect_replies
       end
@@ -42,8 +41,7 @@ module NNQ
         loop do
           body = @sock.receive
           break if body.nil?
-          reply = @fmt.decompress([body])
-          reply = [Marshal.load(reply.first)] if config.format == :marshal
+          reply = config.format == :marshal ? Marshal.load(body) : body
           output(eval_recv_expr(reply))
         rescue NNQ::TimedOut
           break
@@ -83,7 +81,7 @@ module NNQ
           reply = eval_recv_expr(msg)
           unless reply.equal?(SENT)
             output(reply)
-            send_reply(reply || [""])
+            send_reply(reply || "")
           end
         elsif config.echo
           output(msg)
@@ -101,10 +99,9 @@ module NNQ
 
 
       def send_reply(msg)
-        return if msg.empty?
-        msg = [Marshal.dump(msg.first)] if config.format == :marshal
-        msg = @fmt.compress(msg)
-        @sock.send_reply(msg.first)
+        return if msg.nil?
+        body = config.format == :marshal ? Marshal.dump(msg) : msg
+        @sock.send_reply(body)
         transient_ready!
       end
     end

data/lib/nnq/cli/term.rb

@@ -36,9 +36,9 @@ module NNQ
         prefix = log_prefix(verbose)
         case event.type
         when :message_sent
-          "#{prefix}nnq: >> #{Formatter.preview([event.detail[:body]])}"
+          "#{prefix}nnq: >> #{Formatter.preview(event.detail[:body])}"
         when :message_received
-          "#{prefix}nnq: << #{Formatter.preview([event.detail[:body]])}"
+          "#{prefix}nnq: << #{Formatter.preview(event.detail[:body])}"
         else
           ep     = event.endpoint ? " #{event.endpoint}" : ""
           detail = event.detail ? " #{event.detail}" : ""

data/lib/nnq/cli/version.rb

@@ -2,6 +2,6 @@
 
 module NNQ
   module CLI
-    VERSION = "0.2.0"
+    VERSION = "0.4.0"
   end
 end

data/lib/nnq/cli.rb

@@ -1,6 +1,17 @@
 # frozen_string_literal: true
 
 require "optparse"
+
+# Forward-declare NNQ::Zstd::ProtocolError so the rescue clause below
+# resolves even when compression wasn't requested and `nnq/zstd` was
+# never required. The real class is defined in nnq-zstd; re-opening it
+# here with the same StandardError superclass is benign.
+module NNQ
+  module Zstd
+    class ProtocolError < StandardError; end
+  end
+end
+
 require_relative "cli/version"
 require_relative "cli/config"
 require_relative "cli/cli_parser"
@@ -153,8 +164,8 @@ module NNQ
                    runner_class.new(config)
                  end
         runner.call(task)
-      rescue DecompressError => e
-        $stderr.puts "nnq: #{e.message}"
+      rescue NNQ::Zstd::ProtocolError => e
+        $stderr.puts "nnq: zstd protocol error: #{e.message}"
         exit 1
       rescue IO::TimeoutError, Async::TimeoutError
         $stderr.puts "nnq: timeout" unless config.quiet

metadata

@@ -1,7 +1,7 @@
 --- !ruby/object:Gem::Specification
 name: nnq-cli
 version: !ruby/object:Gem::Version
-  version: 0.2.0
+  version: 0.4.0
 platform: ruby
 authors:
 - Patrik Wenger
@@ -24,37 +24,37 @@ dependencies:
       - !ruby/object:Gem::Version
         version: '0.5'
 - !ruby/object:Gem::Dependency
-  name: msgpack
+  name: nnq-zstd
   requirement: !ruby/object:Gem::Requirement
     requirements:
-    - - ">="
+    - - "~>"
       - !ruby/object:Gem::Version
-        version: '0'
+        version: '0.1'
   type: :runtime
   prerelease: false
   version_requirements: !ruby/object:Gem::Requirement
     requirements:
-    - - ">="
+    - - "~>"
       - !ruby/object:Gem::Version
-        version: '0'
+        version: '0.1'
 - !ruby/object:Gem::Dependency
-  name: rlz4
+  name: msgpack
   requirement: !ruby/object:Gem::Requirement
     requirements:
-    - - "~>"
+    - - ">="
       - !ruby/object:Gem::Version
-        version: '0.1'
+        version: '0'
   type: :runtime
   prerelease: false
   version_requirements: !ruby/object:Gem::Requirement
     requirements:
-    - - "~>"
+    - - ">="
       - !ruby/object:Gem::Version
-        version: '0.1'
+        version: '0'
 description: Command-line tool for sending and receiving nanomsg SP messages on any
   NNQ socket type (REQ/REP, PUB/SUB, PUSH/PULL, PAIR). Supports Ruby eval (-e/-E),
   script handlers (-r), the virtual `pipe` socket with optional Ractor parallelism,
-  multiple formats (ASCII, JSON Lines, msgpack, Marshal), and LZ4 compression. Like
+  multiple formats (ASCII, JSON Lines, msgpack, Marshal), and Zstd compression. Like
   nngcat from libnng, but with Ruby superpowers.
 email:
 - paddor@gmail.com