omq-cli 0.15.1 → 0.15.3

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: '08825b7e8c6acff6b7c8847cd1966def9bfa751a06cfb0a69c36f0bdc8f71b0e'
-  data.tar.gz: 2d060c1e4bdfea82a807f053ff5457eb392d372fd73e1c4f962b4b90cbbcfa06
+  metadata.gz: 5b503b668b3427210a067278ba175bbda172e98ccf19df42f4dcf4cc84cd1dd4
+  data.tar.gz: 9861f0dd910a0ca723b97043996a8c5e47c2222efd77321f8f1e43efdc872818
 SHA512:
-  metadata.gz: 5784667a548da85592c289fa905c70290e8e90426815a2fafbf5b833f15159c4d0bdaf1ebb561d50dfe4ed4e801f2e90ffd44c04541a2e62a59e6d17da6f44ef
-  data.tar.gz: ff42a632e0360a1b926d7833cbbeaa3331a2db0d10a47d0d4b5b5f7ef863d948a58cf5972643872566478c2ca0706b9394f75e49518887dc66ad54b67140d9bc
+  metadata.gz: 82972297897d9ea36f3a10ecad3372b476ae935e9b4d20ff312f23db5ecee438b52b0a443d814048f4dab8d5e5243e927509a994917b4926058d80c3f314188b
+  data.tar.gz: cbca40c66d0061a3d51782e3b1cf001aad9d44596f2b9203a7d6837c087049227565456f07646de7e2788d827ce91a1ba5290fa7c8792f103475b867ee663c28

data/CHANGELOG.md

@@ -1,5 +1,58 @@
 # Changelog
 
+## 0.15.3 — 2026-04-16
+
+### Changed
+
+- **Cache decoded `--data` input.** `read_inline_data` and
+  `ParallelWorker#compute_reply` now memoize the decoded result
+  instead of re-decoding the same literal on every iteration.
+
+## Unreleased
+
+### Added
+
+- **REQ generator mode (`-E`/`-e` with no stdin).** `omq req` now
+  produces each request from the send-eval alone when no `-d`/`-f`
+  is given and stdin is not piped, matching the existing PUSH/PUB
+  behaviour. Bounded by `-n` or paced by `-i` like the other
+  generator-capable runners.
+
+### Tests
+
+- **System tests for REQ and PUB `-E` generator mode.** REQ fires
+  `-E'"foo"' -n 3` against a REP running `-e '|(a)| a.upcase' -n 3`
+  and verifies three `FOO` replies round-trip. PUB fires
+  `-E'"tick"' -i 0.05 -n 3` against a SUB and verifies three `tick`
+  frames are received.
+- **System tests split into themed files under `test/system/`.** The
+  monolithic `test/system_test.sh` is replaced by 12 standalone files
+  (req_rep, push_pull, pub_sub, router_dealer, format, compression,
+  interval, transport, script, validation, pipe, parallel) sharing
+  `support.sh` helpers. `run_all.sh` chains them; each file also runs
+  standalone. `rake test:system` now invokes `run_all.sh`.
+
+## 0.15.2 — 2026-04-15
+
+### Fixed
+
+- **`-vvv` trace output for REQ/REP/PAIR.** `recv_msg` now calls
+  `trace_recv` itself, so every runner logs received messages under
+  `-vvv` (previously only `run_recv_logic` / `recv_tick` runners did,
+  leaving REQ/REP/PAIR receive-side traces silent).
+
+### Tests
+
+- **System test for REQ/REP verbose trace order.** Asserts REQ logs
+  `>> hi` then `<< HI`, and REP (with `-e'it.first.upcase'`) logs
+  `<< hi` then `>> HI`.
+- **Test suite aborts on background-thread exceptions.**
+  `Thread.abort_on_exception = true` in `test_helper.rb` — a raising
+  spawned thread now re-raises in the main thread immediately instead
+  of leaving the test hanging on a receive from a dead peer.
+- **`connect_before_bind_test`**: drop `linger: 1` from
+  `OMQ::PULL.new` — PULL is read-only and doesn't accept `linger`.
+
 ## 0.15.0 — 2026-04-14
 
 ### Changed

data/README.md

@@ -1,33 +1,58 @@
-# omq — ZeroMQ CLI
+# omq — Swiss army knife for ØMQ
 
 [![Gem Version](https://img.shields.io/gem/v/omq-cli?color=e9573f)](https://rubygems.org/gems/omq-cli)
 [![License: ISC](https://img.shields.io/badge/License-ISC-blue.svg)](LICENSE)
 [![Ruby](https://img.shields.io/badge/Ruby-%3E%3D%203.3-CC342D?logo=ruby&logoColor=white)](https://www.ruby-lang.org)
 
-Command-line tool for sending and receiving ZeroMQ messages on any socket type.
-Like `nngcat` from libnng, but with Ruby eval, Ractor parallelism, and message handlers.
-
-Built on [omq](https://github.com/zeromq/omq) — pure Ruby ZeroMQ, no C dependencies.
-
-## Install
+A command-line tool that speaks every ZeroMQ socket pattern, transforms
+messages with inline Ruby, parallelizes across Ractors, encrypts with CURVE,
+and reshapes stdin/stdout through three I/O formats on top of three
+on-the-wire formats. Built on [ØMQ](https://github.com/zeromq/omq) — pure
+Ruby, no C dependencies.
 
 ```sh
 gem install omq-cli
 ```
 
-## Quick Start
+Think of it as `nngcat` with a scripting engine bolted on. A few things it can do out of the box:
 
 ```sh
-# Echo server
+# an echo server in one line
 omq rep -b tcp://:5555 --echo
 
-# Client
-echo "hello" | omq req -c tcp://localhost:5555
-
-# Upcase server — -e evals Ruby on each incoming message
+# upcase every incoming message
 omq rep -b tcp://:5555 -e 'it.map(&:upcase)'
+
+# publish a live timestamp every second
+omq pub -c tcp://localhost:5556 -E 'Time.now.to_s' -i 1
+
+# CPU-parallel PULL → transform → PUSH pipeline across all cores
+omq pipe -c@work -c@sink -P0 -r./fib.rb -e 'fib(Integer(it.first)).to_s'
+
+# JSON over the wire, filter by field
+omq sub -c tcp://localhost:5556 -rjson -e 'JSON.parse(it.first)["temperature"]'
 ```
 
+## Highlights
+
+- **Every socket pattern** — req/rep, pub/sub, push/pull, dealer/router,
+  xpub/xsub, pair, and draft types (client/server, radio/dish, scatter/gather,
+  peer, channel)
+- **Inline Ruby transforms** — `-e` rewrites incoming messages, `-E` rewrites
+  outgoing. `next` / `break` / `nil` for flow control, `BEGIN{}` / `END{}` for
+  awk-style aggregation, `-r` to load helper scripts
+- **Ractor-parallel `pipe`** — `-P0` spawns one worker Ractor per core, each
+  with its own PULL/PUSH pair. CPU-bound transforms actually scale
+- **I/O formats** — ASCII, quoted, or JSONL for stdin/stdout (display only;
+  wire stays plain)
+- **Wire formats** — raw ZMTP, MessagePack, or Ruby Marshal shape the frame
+  payload itself, so arbitrary Ruby objects round-trip end-to-end. Optional
+  Zstandard compression (`-z`) on top
+- **CURVE encryption** — end-to-end encrypted sockets via libsodium (or nuckle,
+  pure Ruby). `omq keygen` generates a persistent keypair
+- **Transient mode** — `--transient` exits cleanly when peers disconnect,
+  perfect for pipeline workers and one-shot sinks
+
 ```
 Usage: omq TYPE [options]
 
@@ -61,7 +86,8 @@ Pipe takes two positional endpoints (input, output) or uses `--in`/`--out` for m
 | `scatter` | `gather` | Pipeline (draft, single-frame only) |
 | `radio` | `dish` | Group messaging (draft, single-frame only) |
 
-Send-only sockets read from stdin (or `--data`/`--file`) and send. Recv-only sockets receive and write to stdout.
+Send-only sockets read from stdin (or `--data`/`--file`) and send. Recv-only
+sockets receive and write to stdout.
 
 ```sh
 echo "task" | omq push -c tcp://worker:5557
@@ -96,8 +122,8 @@ echo "hello" | omq req -c tcp://localhost:5555
 | `dealer` | Like `pair` but round-robin send to multiple peers |
 | `channel` | Like `pair` (draft, single-frame) |
 
-These spawn two concurrent tasks: a receiver (prints incoming) and a sender (reads stdin).
-`-e` transforms incoming, `-E` transforms outgoing.
+These spawn two concurrent tasks: a receiver (prints incoming) and a sender
+(reads stdin). `-e` transforms incoming, `-E` transforms outgoing.
 
 ### Routing sockets
 
@@ -212,8 +238,7 @@ echo hello | omq push -c tcp://localhost:5557 -E 'it.map(&:upcase)'
 omq pull -b tcp://:5557 -e 'it.first.include?("error") ? it : nil'
 
 # REQ: different transforms per direction
-echo hello | omq req -c tcp://localhost:5555 \
-  -E 'it.map(&:upcase)' -e 'it.map(&:reverse)'
+echo hello | omq req -c tcp://localhost:5555 -E 'it.map(&:upcase)' -e 'it.map(&:reverse)'
 
 # generate messages without stdin
 omq pub -c tcp://localhost:5556 -E 'Time.now.to_s' -i 1
@@ -305,17 +330,32 @@ OMQ.outgoing { |msg| [*msg, Time.now.iso8601] }
 
 ## Formats
 
+Two distinct axes: **I/O formats** reshape how messages are read from stdin
+and printed to stdout, while **wire formats** shape the frame payload that
+actually goes over ZMTP. Pick one of each — they compose freely.
+
+### I/O formats (stdin/stdout only)
+
 | Flag | Format |
 |------|--------|
 | `-A` / `--ascii` | Tab-separated frames, non-printable → dots (default) |
 | `-Q` / `--quoted` | C-style escapes, lossless round-trip |
-| `--raw` | Raw ZMTP binary (pipe to `hexdump -C` for debugging) |
 | `-J` / `--jsonl` | JSON Lines — `["frame1","frame2"]` per line |
-| `--msgpack` | MessagePack arrays (binary stream) |
-| `-M` / `--marshal` | Ruby Marshal — one arbitrary Ruby object per message |
 
-Multipart messages: in ASCII/quoted mode, frames are tab-separated. In JSONL mode,
-each message is a JSON array.
+Display-only: the wire carries plain ZMTP frames. Multipart messages are
+tab-separated in ASCII/quoted mode and encoded as JSON arrays in JSONL.
+
+### Wire formats (on the ZMTP frame payload)
+
+| Flag | Format |
+|------|--------|
+| `--raw` | Raw ZMTP binary (pipe to `hexdump -C` for debugging) |
+| `--msgpack` | MessagePack — each frame is one packed object |
+| `-M` / `--marshal` | Ruby Marshal — each frame is one arbitrary Ruby object |
+
+Wire formats reshape the payload end-to-end: inside `-e`/`-E`, `it` is the
+decoded object (not an Array of frames), so scalars, hashes, and custom
+classes flow through transparently between peers speaking the same format.
 
 ```sh
 # send multipart via tabs
@@ -326,12 +366,8 @@ echo '["key","value"]' | omq push -c tcp://localhost:5557 -J
 omq pull -b tcp://:5557 -J
 ```
 
-Under `-M`, each wire frame is one Marshal-dumped Ruby object. Inside `-e` / `-E`,
-`it` is that raw object — not an Array of frames — so scalars, hashes, custom
-classes, or any Marshal-safe value flow through transparently:
-
 ```sh
-# send a bare String, receive a { string => encoding } Hash
+# send a bare String with Marshal, receive a { string => encoding } Hash
 omq push -b tcp://:5557 -ME '"foo"'
 omq pull -c tcp://:5557 -M -e '{it => it.encoding}'
 # => {"foo" => #<Encoding:UTF-8>}
@@ -413,8 +449,7 @@ omq rep -b tcp://:5555 --echo --curve-server --crypto nuckle
 omq rep -b tcp://:5555 --echo --curve-server
 
 # client (paste the key)
-echo "secret" | omq req -c tcp://localhost:5555 \
-  --curve-server-key '<key from server>'
+echo "secret" | omq req -c tcp://localhost:5555 --curve-server-key '<key from server>'
 ```
 
 Persistent keys via env vars: `OMQ_SERVER_PUBLIC` + `OMQ_SERVER_SECRET` (server), `OMQ_SERVER_KEY` (client).

data/lib/omq/cli/base_runner.rb

@@ -289,7 +289,6 @@ module OMQ
           loop do
             parts = recv_msg
             break if parts.nil?
-            trace_recv(parts)
             parts = eval_recv_expr(parts)
             output(parts)
             i += 1
@@ -316,7 +315,6 @@ module OMQ
           @recv_tick_eof = true
           return 0
         end
-        trace_recv(parts)
         parts = eval_recv_expr(parts)
         output(parts)
         1
@@ -405,6 +403,7 @@ module OMQ
           parts = Marshal.load(parts.first)
         end
 
+        trace_recv(parts)
         transient_ready!
         parts
       end
@@ -423,7 +422,7 @@ module OMQ
 
       def read_inline_data
         if config.data
-          @fmt.decode(config.data + "\n")
+          @inline_data ||= @fmt.decode(config.data + "\n")
         else
           @file_data ||= (config.file == "-" ? $stdin.read : File.read(config.file)).chomp
           @fmt.decode(@file_data + "\n")

data/lib/omq/cli/expression_evaluator.rb

@@ -28,6 +28,7 @@ module OMQ
           expr, begin_body, end_body = extract_blocks(src)
           @begin_proc = eval("proc { #{begin_body} }") if begin_body
           @end_proc   = eval("proc { #{end_body} }")   if end_body
+
           if expr && !expr.strip.empty?
             @eval_proc = eval("proc { #{expr} }")
           end
@@ -85,6 +86,7 @@ module OMQ
         begin_proc = eval("proc { #{begin_body} }") if begin_body
         end_proc   = eval("proc { #{end_body} }")   if end_body
         eval_proc  = nil
+
         if expr && !expr.strip.empty?
           eval_proc = eval("proc { #{expr} }")
         end
@@ -100,12 +102,12 @@ module OMQ
       # (Ractors cannot call back into instance state).
       #
       def self.extract_block(expr, keyword)
-        start = expr.index(/#{keyword}\s*\{/)
-        return [expr, nil] unless start
+        start = expr.index(/#{keyword}\s*\{/) or return [expr, nil]
 
         i     = expr.index("{", start)
         depth = 1
         j     = i + 1
+
         while j < expr.length && depth > 0
           case expr[j]
           when "{"
@@ -113,6 +115,7 @@ module OMQ
           when "}"
             depth -= 1
           end
+
           j += 1
         end
 

data/lib/omq/cli/parallel_worker.rb

@@ -157,7 +157,7 @@ module OMQ
         elsif @config.echo
           parts
         elsif @config.data
-          @fmt.decode(@config.data + "\n")
+          @inline_data ||= @fmt.decode(@config.data + "\n")
         else
           parts
         end

data/lib/omq/cli/req_rep.rb

@@ -10,19 +10,27 @@ module OMQ
       def run_loop(task)
         n = config.count
         i = 0
-        sleep(config.delay) if config.delay
+
+        sleep config.delay if config.delay
+        generator = @send_eval_proc && !config.data && !config.file && !stdin_ready?
+
         loop do
-          parts = read_next
-          break unless parts
-          parts = eval_send_expr(parts)
+          if generator
+            parts = eval_send_expr(nil)
+          else
+            parts = read_next
+            break unless parts
+            parts = eval_send_expr(parts)
+          end
+
           next unless parts
+
           send_msg(parts)
-          reply = recv_msg
-          break if reply.nil?
+          reply = recv_msg or break
           output(eval_recv_expr(reply))
           i += 1
           break if n && n > 0 && i >= n
-          break if !config.interval && (config.data || config.file)
+          break if !config.interval && (generator || config.data || config.file) && !(n && n > 0)
           wait_for_interval if config.interval
         end
       end
@@ -38,7 +46,11 @@ module OMQ
     # Runner for REP sockets (synchronous request-reply server).
     class RepRunner < BaseRunner
       def call(task)
-        config.parallel ? run_parallel_workers(:REP) : super
+        if config.parallel
+          run_parallel_workers(:REP)
+        else
+          super
+        end
       end
 
 
@@ -48,9 +60,9 @@ module OMQ
       def run_loop(task)
         n = config.count
         i = 0
+
         loop do
-          msg = recv_msg
-          break if msg.nil?
+          msg = recv_msg or break
           break unless handle_rep_request(msg)
           i += 1
           break if n && n > 0 && i >= n
@@ -61,6 +73,7 @@ module OMQ
       def handle_rep_request(msg)
         if config.recv_expr || @recv_eval_proc
           reply = eval_recv_expr(msg)
+
           unless reply.equal?(SENT)
             output(reply)
             send_msg(reply || [""])
@@ -76,6 +89,7 @@ module OMQ
         else
           abort "REP needs a reply source: --echo, --data, --file, -e, or stdin pipe"
         end
+
         true
       end
     end

data/lib/omq/cli/version.rb

@@ -2,6 +2,6 @@
 
 module OMQ
   module CLI
-    VERSION = "0.15.1"
+    VERSION = "0.15.3"
   end
 end

metadata

@@ -1,7 +1,7 @@
 --- !ruby/object:Gem::Specification
 name: omq-cli
 version: !ruby/object:Gem::Version
-  version: 0.15.1
+  version: 0.15.3
 platform: ruby
 authors:
 - Patrik Wenger
@@ -15,20 +15,14 @@ dependencies:
     requirements:
     - - "~>"
       - !ruby/object:Gem::Version
-        version: '0.19'
-    - - ">="
-      - !ruby/object:Gem::Version
-        version: 0.19.3
+        version: '0.22'
   type: :runtime
   prerelease: false
   version_requirements: !ruby/object:Gem::Requirement
     requirements:
     - - "~>"
       - !ruby/object:Gem::Version
-        version: '0.19'
-    - - ">="
-      - !ruby/object:Gem::Version
-        version: 0.19.3
+        version: '0.22'
 - !ruby/object:Gem::Dependency
   name: omq-ffi
   requirement: !ruby/object:Gem::Requirement
@@ -49,84 +43,84 @@ dependencies:
     requirements:
     - - "~>"
       - !ruby/object:Gem::Version
-        version: '0.1'
+        version: '0.2'
   type: :runtime
   prerelease: false
   version_requirements: !ruby/object:Gem::Requirement
     requirements:
     - - "~>"
       - !ruby/object:Gem::Version
-        version: '0.1'
+        version: '0.2'
 - !ruby/object:Gem::Dependency
   name: omq-rfc-radiodish
   requirement: !ruby/object:Gem::Requirement
     requirements:
     - - "~>"
       - !ruby/object:Gem::Version
-        version: '0.1'
+        version: '0.2'
   type: :runtime
   prerelease: false
   version_requirements: !ruby/object:Gem::Requirement
     requirements:
     - - "~>"
       - !ruby/object:Gem::Version
-        version: '0.1'
+        version: '0.2'
 - !ruby/object:Gem::Dependency
   name: omq-rfc-scattergather
   requirement: !ruby/object:Gem::Requirement
     requirements:
     - - "~>"
       - !ruby/object:Gem::Version
-        version: '0.1'
+        version: '0.2'
   type: :runtime
   prerelease: false
   version_requirements: !ruby/object:Gem::Requirement
     requirements:
     - - "~>"
       - !ruby/object:Gem::Version
-        version: '0.1'
+        version: '0.2'
 - !ruby/object:Gem::Dependency
   name: omq-rfc-channel
   requirement: !ruby/object:Gem::Requirement
     requirements:
     - - "~>"
       - !ruby/object:Gem::Version
-        version: '0.1'
+        version: '0.2'
   type: :runtime
   prerelease: false
   version_requirements: !ruby/object:Gem::Requirement
     requirements:
     - - "~>"
       - !ruby/object:Gem::Version
-        version: '0.1'
+        version: '0.2'
 - !ruby/object:Gem::Dependency
   name: omq-rfc-p2p
   requirement: !ruby/object:Gem::Requirement
     requirements:
     - - "~>"
       - !ruby/object:Gem::Version
-        version: '0.1'
+        version: '0.2'
   type: :runtime
   prerelease: false
   version_requirements: !ruby/object:Gem::Requirement
     requirements:
     - - "~>"
       - !ruby/object:Gem::Version
-        version: '0.1'
+        version: '0.2'
 - !ruby/object:Gem::Dependency
   name: omq-rfc-zstd
   requirement: !ruby/object:Gem::Requirement
     requirements:
     - - "~>"
       - !ruby/object:Gem::Version
-        version: '0.2'
+        version: '0.3'
   type: :runtime
   prerelease: false
   version_requirements: !ruby/object:Gem::Requirement
     requirements:
     - - "~>"
       - !ruby/object:Gem::Version
-        version: '0.2'
+        version: '0.3'
 - !ruby/object:Gem::Dependency
   name: msgpack
   requirement: !ruby/object:Gem::Requirement