omq 0.8.0 → 0.9.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 852e5a45a7b9f61b32004aa2079e1289291c3ac2ecc5efeb78911d88d3b63b2e
-  data.tar.gz: b475a15807b8e98d4a852ec78b07ecaa4a0b15e68d3aec2fd9b2702533cb5d5f
+  metadata.gz: a14af693117ccfb193f54e9bc2a4faae9c2bf6396e7769abee97f279fd62fdba
+  data.tar.gz: 1f5e99ae3b61402d2b9e506c84fdb07d2003695515d3f736159cf2c373cc39d0
 SHA512:
-  metadata.gz: 0afb8f440937ae11bebe4ce3ecc1df599103f27300ed721b14437d708bd1cba1cd5b0aeb6873bd7281d054c42f6353abc29b1c8c542f4708152c2ad6b047b5f0
-  data.tar.gz: b3477d54c520f7ed09dc4647b84841c9937d863036c4301afc3110bd959fd794bbd91db56d0204f317cd66ef62bf63ad8b1772521dc48f96912ae01e9bea8f81
+  metadata.gz: 76539e2a9fbdcbe59590e1fa499e300b8f7ad3d527255af6f94665139c3c0a60151f0b82202df372df1cdbc16e90f46369aa7b159342f951cf52babc897b837c
+  data.tar.gz: dba241d3283a0a0330ba0d12081c08f3ae074acaa8be7b1e9292dee9c19a483d822f896a4442ec585005dbeb36efa704b87f51c53e07c1de0ede54a85150f55c

data/CHANGELOG.md

@@ -1,5 +1,25 @@
 # Changelog
 
+## 0.9.0 — 2026-03-31
+
+### Breaking
+
+- **CLI extracted into omq-cli gem** — the `omq` executable, all CLI
+  code (`lib/omq/cli/`), tests, and `CLI.md` have moved to the
+  [omq-cli](https://github.com/paddor/omq-cli) gem. `gem install omq`
+  no longer provides the `omq` command — use `gem install omq-cli`.
+- **`OMQ.outgoing` / `OMQ.incoming`** registration API moved to omq-cli.
+  Library-only users are unaffected (these were CLI-specific).
+
+### Changed
+
+- **Gemspec is library-only** — no `exe/`, no `bindir`, no `executables`.
+- **README** — restored title, replaced inline CLI section with a
+  pointer to omq-cli, fixed ZMTP attribution for protocol-zmtp.
+- **DESIGN.md** — acknowledged protocol-zmtp, clarified transient
+  task / linger interaction, removed ZMTP wire protocol section (now in
+  protocol-zmtp), simplified inproc description, removed CLI section.
+
 ## 0.8.0 — 2026-03-31
 
 ### Breaking

data/README.md

@@ -1,4 +1,4 @@
-# OMQ — ZeroMQ in pure Ruby
+# OMQ — Where did the C dependency go!?
 
 [![CI](https://github.com/zeromq/omq/actions/workflows/ci.yml/badge.svg)](https://github.com/zeromq/omq/actions/workflows/ci.yml)
 [![Gem Version](https://img.shields.io/gem/v/omq?color=e9573f)](https://rubygems.org/gems/omq)
@@ -7,7 +7,7 @@
 
 `gem install omq` — that's it. No libzmq, no compiler, no system packages. Just Ruby.
 
-OMQ implements the [ZMTP 3.1](https://rfc.zeromq.org/spec/23/) wire protocol from scratch using [Async](https://github.com/socketry/async) fibers. It speaks native ZeroMQ on the wire and interoperates with libzmq, pyzmq, CZMQ, and everything else in the ZMQ ecosystem.
+OMQ builds ZeroMQ socket patterns on top of [protocol-zmtp](https://github.com/paddor/protocol-zmtp) (a pure Ruby [ZMTP 3.1](https://rfc.zeromq.org/spec/23/) codec) using [Async](https://github.com/socketry/async) fibers. It speaks native ZeroMQ on the wire and interoperates with libzmq, pyzmq, CZMQ, and everything else in the ZMQ ecosystem.
 
 > **234k msg/s** inproc | **49k msg/s** ipc | **36k msg/s** tcp
 >
@@ -31,12 +31,12 @@ See [GETTING_STARTED.md](GETTING_STARTED.md) for a ~30 min walkthrough of all ma
 
 - **Zero dependencies on C** — no extensions, no FFI, no libzmq. `gem install` just works everywhere
 - **Fast** — YJIT-optimized hot paths, batched sends, 234k msg/s inproc with 12 µs latency
-- **`omq` CLI** — pipe, filter, and transform messages from the terminal with Ruby eval, Ractor parallelism, and [script handlers](CLI.md#script-handlers--r)
+- **[`omq` CLI](https://github.com/paddor/omq-cli)** — `gem install omq-cli` for a command-line tool with Ruby eval, Ractor parallelism, and script handlers
 - **Every socket pattern** — req/rep, pub/sub, push/pull, dealer/router, xpub/xsub, pair, and all draft types
 - **Every transport** — tcp, ipc (Unix domain sockets), inproc (in-process queues)
 - **Async-native** — built on fibers, non-blocking from the ground up. A shared IO thread handles sockets outside of Async — no reactor needed for simple scripts
 - **Wire-compatible** — interoperates with libzmq, pyzmq, CZMQ over tcp and ipc
-- **Bind/connect order doesn't matter** — connect before bind, bind before connect, peers come and go. ZeroMQ reconnects and requeues automatically
+- **Bind/connect order doesn't matter** — connect before bind, bind before connect, peers come and go. ZeroMQ reconnects automatically and queued messages drain when peers arrive
 
 For architecture internals, see [DESIGN.md](DESIGN.md).
 
@@ -150,58 +150,18 @@ All sockets are thread-safe. Default HWM is 1000 messages per socket. Classes li
 | **PEER** | Routing-ID | Fair-queue | Block |
 | **CHANNEL** | Exclusive 1-to-1 | Exclusive 1-to-1 | Block |
 
-## omq — CLI tool
+## CLI
 
-`omq` is a command-line tool for sending and receiving messages on any OMQ socket. Like `nngcat` from libnng, but with Ruby superpowers.
+Install [omq-cli](https://github.com/paddor/omq-cli) for a command-line tool that sends, receives, pipes, and transforms ZeroMQ messages from the terminal:
 
 ```sh
-# Echo server
-omq rep -b tcp://:5555 --echo
-
-# Upcase server — -e evals Ruby on each incoming message
-omq rep -b tcp://:5555 -e '$F.map(&:upcase)'
+gem install omq-cli
 
-# Client
+omq rep -b tcp://:5555 --echo
 echo "hello" | omq req -c tcp://localhost:5555
-# => HELLO
-
-# PUB/SUB
-omq sub -b tcp://:5556 -s "weather."  &
-echo "weather.nyc 72F" | omq pub -c tcp://localhost:5556 -d 0.3
-
-# Pipeline with filtering
-tail -f /var/log/syslog | omq push -c tcp://collector:5557
-omq pull -b tcp://:5557 -e 'next unless /error/; $F'
-
-# Transform outgoing messages with -E
-echo hello | omq push -c tcp://localhost:5557 -E '$F.map(&:upcase)'
-
-# REQ: transform request and reply independently
-echo hello | omq req -c tcp://localhost:5555 \
-  -E '$F.map(&:upcase)' -e '$F.map(&:reverse)'
-
-# Pipe: PULL → eval → PUSH in one process
-omq pipe -c ipc://@work -c ipc://@sink -e '$F.map(&:upcase)'
-
-# Pipe with Ractor workers for CPU parallelism (-P = all CPUs)
-omq pipe -c ipc://@work -c ipc://@sink -P -r./fib -e 'fib(Integer($_)).to_s'
-```
-
-`-e` (recv-eval) transforms incoming messages, `-E` (send-eval) transforms outgoing messages. `$F` is the message parts array, `$_` is the first part. Use `-r` to require gems or load scripts that register handlers via `OMQ.incoming` / `OMQ.outgoing`:
-
-```ruby
-# my_handler.rb
-db = DB.connect("postgres://localhost/app")
-
-OMQ.incoming { db.query($F.first) }
-at_exit { db.close }
-```
-
-```sh
-omq pull -b tcp://:5557 -r./my_handler.rb
 ```
 
-See [CLI.md](CLI.md) for full documentation, or `omq --help` / `omq --examples`.
+See the [omq-cli README](https://github.com/paddor/omq-cli) for full documentation.
 
 ## Development
 

data/lib/omq/version.rb

@@ -1,5 +1,5 @@
 # frozen_string_literal: true
 
 module OMQ
-  VERSION = "0.8.0"
+  VERSION = "0.9.0"
 end

metadata

@@ -1,11 +1,11 @@
 --- !ruby/object:Gem::Specification
 name: omq
 version: !ruby/object:Gem::Version
-  version: 0.8.0
+  version: 0.9.0
 platform: ruby
 authors:
 - Patrik Wenger
-bindir: exe
+bindir: bin
 cert_chain: []
 date: 1980-01-02 00:00:00.000000000 Z
 dependencies:
@@ -51,40 +51,21 @@ dependencies:
     - - "~>"
       - !ruby/object:Gem::Version
         version: '0.11'
-description: Pure Ruby implementation of the ZMTP 3.1 wire protocol (ZeroMQ) with
-  all socket types (REQ/REP, PUB/SUB, PUSH/PULL, DEALER/ROUTER, and draft types) and
-  TCP/IPC/inproc transports. Includes an `omq` CLI for composable message pipelines
-  — pipe, filter, and transform across processes with Ruby eval, multiple formats
-  (JSON, msgpack, marshal), Ractor parallelism, and compression. No native libraries
+description: Pure Ruby implementation of ZeroMQ with all socket types (REQ/REP, PUB/SUB,
+  PUSH/PULL, DEALER/ROUTER, and draft types) and TCP/IPC/inproc transports. Built
+  on protocol-zmtp (ZMTP 3.1 wire protocol) and Async fibers. No native libraries
   required.
 email:
 - paddor@gmail.com
-executables:
-- omq
+executables: []
 extensions: []
 extra_rdoc_files: []
 files:
 - CHANGELOG.md
 - LICENSE
 - README.md
-- exe/omq
 - lib/omq.rb
 - lib/omq/channel.rb
-- lib/omq/cli.rb
-- lib/omq/cli/base_runner.rb
-- lib/omq/cli/channel.rb
-- lib/omq/cli/client_server.rb
-- lib/omq/cli/config.rb
-- lib/omq/cli/formatter.rb
-- lib/omq/cli/pair.rb
-- lib/omq/cli/peer.rb
-- lib/omq/cli/pipe.rb
-- lib/omq/cli/pub_sub.rb
-- lib/omq/cli/push_pull.rb
-- lib/omq/cli/radio_dish.rb
-- lib/omq/cli/req_rep.rb
-- lib/omq/cli/router_dealer.rb
-- lib/omq/cli/scatter_gather.rb
 - lib/omq/client_server.rb
 - lib/omq/pair.rb
 - lib/omq/peer.rb
@@ -148,5 +129,5 @@ required_rubygems_version: !ruby/object:Gem::Requirement
 requirements: []
 rubygems_version: 4.0.6
 specification_version: 4
-summary: Pure Ruby ZMQ library + CLI
+summary: Pure Ruby ZMQ library
 test_files: []

data/exe/omq

@@ -1,6 +0,0 @@
-#!/usr/bin/env ruby
-# frozen_string_literal: true
-Warning[:experimental] = false
-
-require_relative "../lib/omq/cli"
-OMQ::CLI.run

data/lib/omq/cli/base_runner.rb

@@ -1,459 +0,0 @@
-# frozen_string_literal: true
-
-module OMQ
-  module CLI
-    class BaseRunner
-      attr_reader :config, :sock
-
-
-      def initialize(config, socket_class)
-        @config = config
-        @klass  = socket_class
-        @fmt    = Formatter.new(config.format, compress: config.compress)
-      end
-
-
-      def call(task)
-        @sock = create_socket
-        attach_endpoints
-        setup_curve
-        setup_subscriptions
-        compile_expr
-
-        if config.transient
-          start_disconnect_monitor(task)
-          Async::Task.current.yield  # let monitor start waiting
-        end
-
-        sleep(config.delay) if config.delay && config.recv_only?
-        wait_for_peer if needs_peer_wait?
-
-        @sock.instance_exec(&@send_begin_proc) if @send_begin_proc
-        @sock.instance_exec(&@recv_begin_proc) if @recv_begin_proc
-        run_loop(task)
-        @sock.instance_exec(&@send_end_proc) if @send_end_proc
-        @sock.instance_exec(&@recv_end_proc) if @recv_end_proc
-      ensure
-        @sock&.close
-      end
-
-
-      private
-
-
-      # Subclasses override this.
-      def run_loop(task)
-        raise NotImplementedError
-      end
-
-      # ── Socket creation ─────────────────────────────────────────────
-
-
-      def create_socket
-        sock_opts              = { linger: config.linger }
-        sock_opts[:conflate]   = true if config.conflate && %w[pub radio].include?(config.type_name)
-        sock                   = @klass.new(**sock_opts)
-        sock.recv_timeout      = config.timeout if config.timeout
-        sock.send_timeout      = config.timeout if config.timeout
-        sock.reconnect_interval  = config.reconnect_ivl if config.reconnect_ivl
-        sock.heartbeat_interval  = config.heartbeat_ivl if config.heartbeat_ivl
-        sock.identity            = config.identity if config.identity
-        sock.router_mandatory  = true if config.type_name == "router"
-        sock
-      end
-
-
-      def attach_endpoints
-        config.binds.each do |url|
-          @sock.bind(url)
-          log "Bound to #{@sock.last_endpoint}"
-        end
-        config.connects.each do |url|
-          @sock.connect(url)
-          log "Connecting to #{url}"
-        end
-      end
-
-      # ── Peer wait with grace period ─────────────────────────────────
-
-
-      def needs_peer_wait?
-        !config.recv_only? && (config.connects.any? || config.type_name == "router")
-      end
-
-
-      def wait_for_peer
-        with_timeout(config.timeout) do
-          @sock.peer_connected.wait
-          log "Peer connected"
-          if %w[pub xpub].include?(config.type_name)
-            @sock.subscriber_joined.wait
-            log "Subscriber joined"
-          end
-
-          # Grace period: when multiple peers may be connecting (bind or
-          # multiple connect URLs), wait one reconnect interval so
-          # latecomers finish their handshake before we start sending.
-          if config.binds.any? || config.connects.size > 1
-            ri = @sock.options.reconnect_interval
-            sleep(ri.is_a?(Range) ? ri.begin : ri)
-          end
-        end
-      end
-
-      # ── Transient disconnect monitor ────────────────────────────────
-
-
-      def start_disconnect_monitor(task)
-        @transient_barrier = Async::Promise.new
-        task.async do
-          @transient_barrier.wait
-          @sock.all_peers_gone.wait unless @sock.connection_count == 0
-          log "All peers disconnected, exiting"
-          @sock.reconnect_enabled = false
-          if config.send_only?
-            task.stop
-          else
-            @sock.close_read
-          end
-        end
-      end
-
-
-      def transient_ready!
-        if config.transient && !@transient_barrier.resolved?
-          @transient_barrier.resolve(true)
-        end
-      end
-
-      # ── Timeout helper ──────────────────────────────────────────────
-
-
-      def with_timeout(seconds)
-        if seconds
-          Async::Task.current.with_timeout(seconds) { yield }
-        else
-          yield
-        end
-      end
-
-      # ── Socket setup ────────────────────────────────────────────────
-
-
-      def setup_subscriptions
-        case config.type_name
-        when "sub"
-          prefixes = config.subscribes.empty? ? [""] : config.subscribes
-          prefixes.each { |p| @sock.subscribe(p) }
-        when "dish"
-          config.joins.each { |g| @sock.join(g) }
-        end
-      end
-
-
-      def setup_curve
-        server_key_z85 = config.curve_server_key || ENV["OMQ_SERVER_KEY"]
-        server_mode    = config.curve_server || (ENV["OMQ_SERVER_PUBLIC"] && ENV["OMQ_SERVER_SECRET"])
-
-        if server_key_z85
-          if ENV["OMQ_DEV"]
-            require_relative "../../../../omq-curve/lib/omq/curve"
-          else
-            require "omq/curve"
-          end
-          server_key = OMQ::Z85.decode(server_key_z85)
-          client_key = RbNaCl::PrivateKey.generate
-          @sock.mechanism = OMQ::Curve.client(
-            client_key.public_key.to_s, client_key.to_s, server_key: server_key
-          )
-        elsif server_mode
-          if ENV["OMQ_DEV"]
-            require_relative "../../../../omq-curve/lib/omq/curve"
-          else
-            require "omq/curve"
-          end
-          if ENV["OMQ_SERVER_PUBLIC"] && ENV["OMQ_SERVER_SECRET"]
-            server_pub = OMQ::Z85.decode(ENV["OMQ_SERVER_PUBLIC"])
-            server_sec = OMQ::Z85.decode(ENV["OMQ_SERVER_SECRET"])
-          else
-            key        = RbNaCl::PrivateKey.generate
-            server_pub = key.public_key.to_s
-            server_sec = key.to_s
-          end
-          @sock.mechanism = OMQ::Curve.server(server_pub, server_sec)
-          $stderr.puts "OMQ_SERVER_KEY='#{OMQ::Z85.encode(server_pub)}'"
-        end
-      rescue LoadError
-        abort "omq-curve gem required for CURVE encryption: gem install omq-curve"
-      end
-
-      # ── Shared loop bodies ──────────────────────────────────────────
-
-
-      def run_send_logic
-        n = config.count
-        i = 0
-        sleep(config.delay) if config.delay
-        if config.interval
-          i += send_tick
-          unless @send_tick_eof || (n && n > 0 && i >= n)
-            Async::Loop.quantized(interval: config.interval) do
-              i += send_tick
-              break if @send_tick_eof || (n && n > 0 && i >= n)
-            end
-          end
-        elsif config.data || config.file
-          parts = eval_send_expr(read_next)
-          send_msg(parts) if parts
-        elsif stdin_ready?
-          loop do
-            parts = read_next
-            break unless parts
-            parts = eval_send_expr(parts)
-            send_msg(parts) if parts
-            i += 1
-            break if n && n > 0 && i >= n
-          end
-        elsif @send_eval_proc
-          parts = eval_send_expr(nil)
-          send_msg(parts) if parts
-        end
-      end
-
-
-      def send_tick
-        raw = read_next_or_nil
-        if raw.nil? && !@send_eval_proc
-          @send_tick_eof = true
-          return 0
-        end
-        parts = eval_send_expr(raw)
-        send_msg(parts) if parts
-        1
-      end
-
-
-      def run_recv_logic
-        n = config.count
-        i = 0
-        loop do
-          parts = recv_msg
-          break if parts.nil?
-          parts = eval_recv_expr(parts)
-          output(parts)
-          i += 1
-          break if n && n > 0 && i >= n
-        end
-      end
-
-
-      def wait_for_loops(receiver, sender)
-        if config.data || config.file || config.send_expr || config.recv_expr || config.target
-          sender.wait
-          receiver.stop
-        elsif config.count && config.count > 0
-          receiver.wait
-          sender.stop
-        else
-          sender.wait
-          receiver.stop
-        end
-      end
-
-      # ── Message I/O ─────────────────────────────────────────────────
-
-
-      def send_msg(parts)
-        return if parts.empty?
-        parts = [Marshal.dump(parts)] if config.format == :marshal
-        parts = @fmt.compress(parts)
-        @sock.send(parts)
-        transient_ready!
-      end
-
-
-      def recv_msg
-        raw = @sock.receive
-        return nil if raw.nil?
-        parts = @fmt.decompress(raw)
-        parts = Marshal.load(parts.first) if config.format == :marshal
-        transient_ready!
-        parts
-      end
-
-
-      def recv_msg_raw
-        msg = @sock.receive
-        msg&.dup
-      end
-
-
-      def read_next
-        if config.data
-          @fmt.decode(config.data + "\n")
-        elsif config.file
-          @file_data ||= (config.file == "-" ? $stdin.read : File.read(config.file)).chomp
-          @fmt.decode(@file_data + "\n")
-        elsif config.format == :msgpack
-          @fmt.decode_msgpack($stdin)
-        elsif config.format == :marshal
-          @fmt.decode_marshal($stdin)
-        elsif config.format == :raw
-          data = $stdin.read
-          return nil if data.nil? || data.empty?
-          [data]
-        else
-          line = $stdin.gets
-          return nil if line.nil?
-          @fmt.decode(line)
-        end
-      end
-
-
-      def stdin_ready?
-        return @stdin_ready unless @stdin_ready.nil?
-
-        @stdin_ready = !$stdin.closed? &&
-                       !config.stdin_is_tty &&
-                       IO.select([$stdin], nil, nil, 0.01) &&
-                       !$stdin.eof?
-      end
-
-
-      def read_next_or_nil
-        if config.data || config.file
-          read_next
-        elsif @send_eval_proc
-          nil
-        else
-          read_next
-        end
-      end
-
-
-      def output(parts)
-        return if config.quiet || parts.nil?
-        $stdout.write(@fmt.encode(parts))
-        $stdout.flush
-      end
-
-      # ── Routing helpers ─────────────────────────────────────────────
-
-
-      def display_routing_id(id)
-        if id.bytes.all? { |b| b >= 0x20 && b <= 0x7E }
-          id
-        else
-          "0x#{id.unpack1("H*")}"
-        end
-      end
-
-
-      def resolve_target(target)
-        if target.start_with?("0x")
-          [target[2..].delete(" ")].pack("H*")
-        else
-          target
-        end
-      end
-
-      # ── Eval ────────────────────────────────────────────────────────
-
-
-      def compile_expr
-        compile_one_expr(:send, config.send_expr)
-        compile_one_expr(:recv, config.recv_expr)
-        @send_eval_proc ||= wrap_registered_proc(OMQ.outgoing_proc)
-        @recv_eval_proc ||= wrap_registered_proc(OMQ.incoming_proc)
-      end
-
-
-      def wrap_registered_proc(block)
-        return unless block
-        proc do |msg|
-          $_ = msg&.first
-          block.call(msg)
-        end
-      end
-
-
-      def compile_one_expr(direction, src)
-        return unless src
-        expr, begin_body, end_body = extract_blocks(src)
-        instance_variable_set(:"@#{direction}_begin_proc", eval("proc { #{begin_body} }")) if begin_body
-        instance_variable_set(:"@#{direction}_end_proc",   eval("proc { #{end_body} }"))   if end_body
-        if expr && !expr.strip.empty?
-          instance_variable_set(:"@#{direction}_eval_proc", eval("proc { $_ = $F&.first; #{expr} }"))
-        end
-      end
-
-
-      def extract_blocks(expr)
-        begin_body = end_body = nil
-        expr, begin_body = extract_block(expr, "BEGIN")
-        expr, end_body   = extract_block(expr, "END")
-        [expr, begin_body, end_body]
-      end
-
-
-      def extract_block(expr, keyword)
-        start = expr.index(/#{keyword}\s*\{/)
-        return [expr, nil] unless start
-
-        # Find the opening brace
-        i = expr.index("{", start)
-        depth = 1
-        j     = i + 1
-        while j < expr.length && depth > 0
-          case expr[j]
-          when "{" then depth += 1
-          when "}" then depth -= 1
-          end
-          j += 1
-        end
-
-        body    = expr[(i + 1)..(j - 2)]
-        trimmed = expr[0...start] + expr[j..]
-        [trimmed, body]
-      end
-
-
-      SENT = Object.new.freeze # sentinel: eval already sent the reply
-
-      def eval_send_expr(parts)
-        return parts unless @send_eval_proc
-        run_eval(@send_eval_proc, parts)
-      end
-
-
-      def eval_recv_expr(parts)
-        return parts unless @recv_eval_proc
-        run_eval(@recv_eval_proc, parts)
-      end
-
-
-      def run_eval(eval_proc, parts)
-        $F = parts
-        result = @sock.instance_exec(parts, &eval_proc)
-        return nil if result.nil?
-        return SENT if result.equal?(@sock)
-        return [result] if config.format == :marshal
-        case result
-        when Array  then result
-        when String then [result]
-        else             [result.to_str]
-        end
-      rescue => e
-        $stderr.puts "omq: eval error: #{e.message} (#{e.class})"
-        exit 3
-      end
-
-      # ── Logging ─────────────────────────────────────────────────────
-
-
-      def log(msg)
-        $stderr.puts(msg) if config.verbose
-      end
-    end
-  end
-end

data/lib/omq/cli/channel.rb

@@ -1,8 +0,0 @@
-# frozen_string_literal: true
-
-module OMQ
-  module CLI
-    class ChannelRunner < PairRunner
-    end
-  end
-end