omq 0.6.5 → 0.7.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: 94402697e206444e39655869c1382cea2a2839732d373d6cbcc237dcb125cd44
4
- data.tar.gz: dbcaa10eb1213e935325d0290611716552d28394dd479d8fc79564a0af6f0e4d
3
+ metadata.gz: d2e17e1a12c0f44bec5af914b8df2f810029e75c7da17e79e862428a7b0aa99d
4
+ data.tar.gz: 4a8d06116bad67adc4411cb3281302fd6190ecce46ceb47f21cc15a32c3e9006
5
5
  SHA512:
6
- metadata.gz: 9a4b4eab9767b52f874bf85816589558bf42e9730da827220490e80ca9dd499ae5ae604c06022d3db07402042a1b52ea1c88737e2c1556b4d78af90391dc1d2f
7
- data.tar.gz: e2db5e6aaea16af48024be90af8353a81f4d77a4b0b1973131f74dbd915b9aaf52fdd06d08cc57749ffb74b6bd70214ffb2ab04063925887c8a1b72afd4582cb
6
+ metadata.gz: 1dad86f1ad0648690bc5a01d3b7258b89f301d4768bac0831495d27f0a0e3c84f5a30af049732f8cc6885bcd6105654d41708d519da69c30a62739ca58a27418
7
+ data.tar.gz: a71f54aa26dd3e7dea3b249d027c03aeff896c7ded7b0581067fbda609d114c976c8d35a5f7f7e630cbbc34fbb839c7d43f6d445e8f2bb50f1348cb46f8d0e51
data/CHANGELOG.md CHANGED
@@ -1,5 +1,45 @@
1
1
  # Changelog
2
2
 
3
+ ## 0.7.0 — 2026-03-30
4
+
5
+ ### Breaking
6
+
7
+ - **`-e` is now `--recv-eval`** — evaluates incoming messages only.
8
+ Send-only sockets (PUSH, PUB, SCATTER, RADIO) must use `-E` /
9
+ `--send-eval` instead of `-e`.
10
+
11
+ ### Added
12
+
13
+ - **`-E` / `--send-eval`** — eval Ruby for each outgoing message.
14
+ REQ can now transform requests independently from replies.
15
+ ROUTER/SERVER/PEER: `-E` does dynamic routing (first element =
16
+ identity), mutually exclusive with `--target`.
17
+ - **`OMQ.outgoing` / `OMQ.incoming`** — registration API for script
18
+ handlers loaded via `-r`. Blocks receive message parts as a block
19
+ argument (`|msg|`). Setup via closures, teardown via `at_exit`.
20
+ CLI flags override registered handlers.
21
+ - **[CLI.md](CLI.md)** — comprehensive CLI documentation.
22
+ - **[GETTING_STARTED.md](GETTING_STARTED.md)** — renamed from
23
+ `ZGUIDE_SUMMARY.md` for discoverability.
24
+ - **Multi-peer pipe with `--in`/`--out`** — modal switches that assign
25
+ subsequent `-b`/`-c` to the PULL (input) or PUSH (output) side.
26
+ Enables fan-in, fan-out, and mixed bind/connect per side.
27
+ Backward compatible — without `--in`/`--out`, the positional
28
+ 2-endpoint syntax works as before.
29
+
30
+ ### Improved
31
+
32
+ - **YJIT recv pump** — replaced lambda/proc `transform:` parameter in
33
+ `Engine#start_recv_pump` with block captures. No-transform path
34
+ (PUSH/PULL, PUB/SUB) is now branch-free. ~2.5x YJIT speedup on
35
+ inproc, ~2x on ipc/tcp.
36
+
37
+ ### Fixed
38
+
39
+ - **Frozen array from `recv_msg_raw`** — ROUTER/SERVER receiver crashed
40
+ with `FrozenError` when shifting identity off frozen message arrays.
41
+ `#recv_msg_raw` now dups the array.
42
+
3
43
  ## 0.6.5 — 2026-03-30
4
44
 
5
45
  ### Fixed
data/README.md CHANGED
@@ -1,39 +1,44 @@
1
- # OMQ! Where did the C dependency go?!
1
+ # OMQ ZeroMQ in pure Ruby
2
2
 
3
3
  [![CI](https://github.com/zeromq/omq/actions/workflows/ci.yml/badge.svg)](https://github.com/zeromq/omq/actions/workflows/ci.yml)
4
4
  [![Gem Version](https://img.shields.io/gem/v/omq?color=e9573f)](https://rubygems.org/gems/omq)
5
5
  [![License: ISC](https://img.shields.io/badge/License-ISC-blue.svg)](LICENSE)
6
6
  [![Ruby](https://img.shields.io/badge/Ruby-%3E%3D%203.3-CC342D?logo=ruby&logoColor=white)](https://www.ruby-lang.org)
7
7
 
8
- Pure Ruby implementation of the [ZMTP 3.1](https://rfc.zeromq.org/spec/23/) wire protocol ([ZeroMQ](https://zeromq.org/)) using the [Async](https://github.com/socketry/async) gem. No native libraries required.
8
+ `gem install omq` that's it. No libzmq, no compiler, no system packages. Just Ruby.
9
9
 
10
- > **244k msg/s** inproc | **47k msg/s** ipc | **36k msg/s** tcp
10
+ OMQ implements the [ZMTP 3.1](https://rfc.zeromq.org/spec/23/) wire protocol from scratch using [Async](https://github.com/socketry/async) fibers. It speaks native ZeroMQ on the wire and interoperates with libzmq, pyzmq, CZMQ, and everything else in the ZMQ ecosystem.
11
+
12
+ > **234k msg/s** inproc | **49k msg/s** ipc | **36k msg/s** tcp
11
13
  >
12
- > **9 µs** inproc latency | **47 µs** ipc | **61 µs** tcp
14
+ > **12 µs** inproc latency | **51 µs** ipc | **62 µs** tcp
13
15
  >
14
- > Ruby 4.0 + YJIT on a Linux VM on a 2019 MacBook Pro (Intel) — [~340k msg/s with io_uring](bench/README.md#io_uring)
16
+ > Ruby 4.0 + YJIT on a Linux VM see [`bench/`](bench/) for full results
15
17
 
16
18
  ---
17
19
 
18
- ## Highlights
20
+ ## What is ZeroMQ?
21
+
22
+ Brokerless message-oriented middleware. No central server, no extra hop — processes talk directly to each other, cutting latency in half compared to broker-based systems. You get the patterns you'd normally build on top of RabbitMQ or Redis — pub/sub, work distribution, request/reply, fan-out — but decentralized, with no single point of failure.
19
23
 
20
- - **Pure Ruby** no C extensions, no FFI, no libzmq/libczmq dependency
21
- - **All socket types** — req/rep, pub/sub, push/pull, dealer/router, xpub/xsub, pair + draft types (client/server, radio/dish, scatter/gather, peer, channel)
22
- - **Async-native** — built on [Async](https://github.com/socketry/async) fibers, also works with plain threads
23
- - **Ruby-idiomatic API** — messages as `Array<String>`, errors as exceptions, timeouts as `IO::TimeoutError`
24
- - **All transports** — tcp, ipc, inproc
24
+ Networking is hard. ZeroMQ abstracts away reconnection, queuing, load balancing, and framing so you can focus on what your system actually does. Start with threads talking over `inproc://`, split into processes with `ipc://`, scale across machines with `tcp://` — same code, same API, just change the URL.
25
25
 
26
- ## Why pure Ruby?
26
+ If you've ever wired up services with raw TCP, HTTP polling, or Redis pub/sub and wished it was simpler, this is what you've been looking for.
27
27
 
28
- Modern Ruby has closed the gap:
28
+ See [GETTING_STARTED.md](GETTING_STARTED.md) for a ~30 min walkthrough of all major patterns with working code.
29
29
 
30
- - **YJIT** — JIT-compiled hot paths close the throughput gap with C extensions
31
- - **Fiber Scheduler** — non-blocking I/O without callbacks or threads (`Async` builds on this)
32
- - **`io-stream`** — buffered I/O with read-ahead, from the Async ecosystem
30
+ ## Highlights
33
31
 
34
- When [CZTop](https://github.com/paddor/cztop) was written, none of this existed. Today, a pure Ruby ZMTP implementation is fast enough for production use — and you get `gem install` with no compiler toolchain, no system packages, and no segfaults.
32
+ - **Zero dependencies on C** no extensions, no FFI, no libzmq. `gem install` just works everywhere
33
+ - **Fast** — YJIT-optimized hot paths, batched sends, 234k msg/s inproc with 12 µs latency
34
+ - **`omq` CLI** — pipe, filter, and transform messages from the terminal with Ruby eval, Ractor parallelism, and [script handlers](CLI.md#script-handlers--r)
35
+ - **Every socket pattern** — req/rep, pub/sub, push/pull, dealer/router, xpub/xsub, pair, and all draft types
36
+ - **Every transport** — tcp, ipc (Unix domain sockets), inproc (in-process queues)
37
+ - **Async-native** — built on fibers, non-blocking from the ground up. A shared IO thread handles sockets outside of Async — no reactor needed for simple scripts
38
+ - **Wire-compatible** — interoperates with libzmq, pyzmq, CZMQ over tcp and ipc
39
+ - **Bind/connect order doesn't matter** — connect before bind, bind before connect, peers come and go. ZeroMQ reconnects and requeues automatically
35
40
 
36
- See [DESIGN.md](DESIGN.md) for architecture details: task trees, send pump batching, the ZMTP wire protocol, and how OMQ handles the fallacies of distributed computing.
41
+ For architecture internals, see [DESIGN.md](DESIGN.md).
37
42
 
38
43
  ## Install
39
44
 
@@ -45,10 +50,6 @@ gem install omq
45
50
  gem 'omq'
46
51
  ```
47
52
 
48
- ## Learning ZeroMQ
49
-
50
- New to ZeroMQ? See [ZGUIDE_SUMMARY.md](ZGUIDE_SUMMARY.md) — a ~30 min read covering all major patterns with working OMQ code examples.
51
-
52
53
  ## Quick Start
53
54
 
54
55
  ### Request / Reply
@@ -67,7 +68,7 @@ Async do |task|
67
68
  end
68
69
 
69
70
  req << 'hello'
70
- puts req.receive.inspect # => ["HELLO"]
71
+ p req.receive # => ["HELLO"]
71
72
  ensure
72
73
  req&.close
73
74
  rep&.close
@@ -83,7 +84,7 @@ Async do |task|
83
84
  sub.subscribe('') # subscribe to all
84
85
 
85
86
  task.async { pub << 'news flash' }
86
- puts sub.receive.inspect # => ["news flash"]
87
+ p sub.receive # => ["news flash"]
87
88
  ensure
88
89
  pub&.close
89
90
  sub&.close
@@ -98,51 +99,56 @@ Async do
98
99
  pull = OMQ::PULL.bind('inproc://pipeline')
99
100
 
100
101
  push << 'work item'
101
- puts pull.receive.inspect # => ["work item"]
102
+ p pull.receive # => ["work item"]
102
103
  ensure
103
104
  push&.close
104
105
  pull&.close
105
106
  end
106
107
  ```
107
108
 
108
- ## Socket Types
109
+ ### Without Async (IO thread)
109
110
 
110
- | Pattern | Classes | Direction | Multipart |
111
- |---------|---------|-----------|-----------|
112
- | Request/Reply | `REQ`, `REP` | bidirectional | yes |
113
- | Publish/Subscribe | `PUB`, `SUB`, `XPUB`, `XSUB` | unidirectional | yes |
114
- | Pipeline | `PUSH`, `PULL` | unidirectional | yes |
115
- | Routing | `DEALER`, `ROUTER` | bidirectional | yes |
116
- | Exclusive pair | `PAIR` | bidirectional | yes |
117
- | Client/Server | `CLIENT`, `SERVER` | bidirectional | no |
118
- | Group messaging | `RADIO`, `DISH` | unidirectional | no |
119
- | Pipeline (draft) | `SCATTER`, `GATHER` | unidirectional | no |
120
- | Peer-to-peer | `PEER` | bidirectional | no |
121
- | Channel (draft) | `CHANNEL` | bidirectional | no |
122
-
123
- All classes live under `OMQ::`. For the purists, `ØMQ` is an alias:
111
+ OMQ spawns a shared `omq-io` thread when used outside an Async reactor — no boilerplate needed:
124
112
 
125
113
  ```ruby
126
- req = ØMQ::REQ.new(">tcp://localhost:5555")
114
+ require 'omq'
115
+
116
+ push = OMQ::PUSH.bind('tcp://127.0.0.1:5557')
117
+ pull = OMQ::PULL.connect('tcp://127.0.0.1:5557')
118
+
119
+ push << 'hello'
120
+ p pull.receive # => ["hello"]
121
+
122
+ push.close
123
+ pull.close
127
124
  ```
128
125
 
129
- ## Performance
126
+ The IO thread runs all pumps, reconnection, and heartbeating in the background. When you're inside an `Async` block, OMQ uses the existing reactor instead.
130
127
 
131
- Benchmarked with benchmark-ips on Linux x86_64 (Ruby 4.0.2 +YJIT):
128
+ ## Socket Types
132
129
 
133
- #### Throughput (push/pull, 64 B messages)
130
+ All sockets are thread-safe. Default HWM is 1000 messages per socket. Classes live under `OMQ::` (alias: `ØMQ`).
134
131
 
135
- | inproc | ipc | tcp |
136
- |--------|-----|-----|
137
- | 244k/s | 47k/s | 36k/s |
132
+ #### Standard (multipart messages)
138
133
 
139
- #### Latency (req/rep roundtrip)
134
+ | Pattern | Send | Receive | When HWM full |
135
+ |---------|------|---------|---------------|
136
+ | **REQ** / **REP** | Round-robin / route-back | Fair-queue | Block |
137
+ | **PUB** / **SUB** | Fan-out to subscribers | Subscription filter | Drop |
138
+ | **PUSH** / **PULL** | Round-robin to workers | Fair-queue | Block |
139
+ | **DEALER** / **ROUTER** | Round-robin / identity-route | Fair-queue | Block |
140
+ | **XPUB** / **XSUB** | Fan-out (subscription events) | Fair-queue | Drop |
141
+ | **PAIR** | Exclusive 1-to-1 | Exclusive 1-to-1 | Block |
140
142
 
141
- | inproc | ipc | tcp |
142
- |--------|-----|-----|
143
- | 9 µs | 47 µs | 61 µs |
143
+ #### Draft (single-frame only)
144
144
 
145
- See [`bench/`](bench/) for full results and scripts.
145
+ | Pattern | Send | Receive | When HWM full |
146
+ |---------|------|---------|---------------|
147
+ | **CLIENT** / **SERVER** | Round-robin / routing-ID | Fair-queue | Block |
148
+ | **RADIO** / **DISH** | Group fan-out | Group filter | Drop |
149
+ | **SCATTER** / **GATHER** | Round-robin | Fair-queue | Block |
150
+ | **PEER** | Routing-ID | Fair-queue | Block |
151
+ | **CHANNEL** | Exclusive 1-to-1 | Exclusive 1-to-1 | Block |
146
152
 
147
153
  ## omq — CLI tool
148
154
 
@@ -152,7 +158,7 @@ See [`bench/`](bench/) for full results and scripts.
152
158
  # Echo server
153
159
  omq rep -b tcp://:5555 --echo
154
160
 
155
- # Upcase server in one line
161
+ # Upcase server -e evals Ruby on each incoming message
156
162
  omq rep -b tcp://:5555 -e '$F.map(&:upcase)'
157
163
 
158
164
  # Client
@@ -163,50 +169,39 @@ echo "hello" | omq req -c tcp://localhost:5555
163
169
  omq sub -b tcp://:5556 -s "weather." &
164
170
  echo "weather.nyc 72F" | omq pub -c tcp://localhost:5556 -d 0.3
165
171
 
166
- # Pipeline with filtering ($F = message parts, $_ = first part)
167
- # /regexp/ matches against $_, next skips, break stops
172
+ # Pipeline with filtering
168
173
  tail -f /var/log/syslog | omq push -c tcp://collector:5557
169
174
  omq pull -b tcp://:5557 -e 'next unless /error/; $F'
170
175
 
176
+ # Transform outgoing messages with -E
177
+ echo hello | omq push -c tcp://localhost:5557 -E '$F.map(&:upcase)'
178
+
179
+ # REQ: transform request and reply independently
180
+ echo hello | omq req -c tcp://localhost:5555 \
181
+ -E '$F.map(&:upcase)' -e '$F.map(&:reverse)'
182
+
171
183
  # Pipe: PULL → eval → PUSH in one process
172
184
  omq pipe -c ipc://@work -c ipc://@sink -e '$F.map(&:upcase)'
173
185
 
174
186
  # Pipe with Ractor workers for CPU parallelism (-P = all CPUs)
175
187
  omq pipe -c ipc://@work -c ipc://@sink -P -r./fib -e 'fib(Integer($_)).to_s'
188
+ ```
176
189
 
177
- # Exit when all peers disconnect (pipeline workers, sinks)
178
- omq pipe -c ipc://@work -c ipc://@sink --transient -e '$F'
179
-
180
- # JSONL for structured data
181
- echo '["key","value"]' | omq push -c tcp://localhost:5557 -J
182
- omq pull -b tcp://:5557 -J
190
+ `-e` (recv-eval) transforms incoming messages, `-E` (send-eval) transforms outgoing messages. `$F` is the message parts array, `$_` is the first part. Use `-r` to require gems or load scripts that register handlers via `OMQ.incoming` / `OMQ.outgoing`:
183
191
 
184
- # Zstandard compression
185
- omq push -c tcp://remote:5557 -z < data.txt
186
- omq pull -b tcp://:5557 -z
192
+ ```ruby
193
+ # my_handler.rb
194
+ db = DB.connect("postgres://localhost/app")
187
195
 
188
- # CURVE encryption
189
- omq rep -b tcp://:5555 -D "secret" --curve-server
190
- # prints: OMQ_SERVER_KEY='...'
191
- omq req -c tcp://localhost:5555 --curve-server-key '...'
196
+ OMQ.incoming { db.query($F.first) }
197
+ at_exit { db.close }
192
198
  ```
193
199
 
194
- The `-e` flag runs Ruby inside the socket instance — the full socket API (`self <<`, `send`, `subscribe`, ...) is available. `$F` is the message parts array, `$_` is the first part. Use `-r` to require gems:
195
-
196
200
  ```sh
197
- omq sub -c tcp://localhost:5556 -s "" -r json \
198
- -e 'JSON.parse($F.first)["temperature"]'
199
-
200
- # BEGIN/END blocks (like awk) — accumulate and summarize
201
- omq pull -b tcp://:5557 \
202
- -e 'BEGIN{ @sum = 0 } @sum += Integer($_); next END{ puts @sum }'
201
+ omq pull -b tcp://:5557 -r./my_handler.rb
203
202
  ```
204
203
 
205
- Formats: `--ascii` (default, tab-separated), `--quoted`, `--raw`, `--jsonl`, `--msgpack`, `--marshal`. See `omq --help` for all options.
206
-
207
- ## Interop with native ZMQ
208
-
209
- OMQ speaks ZMTP 3.1 on the wire and interoperates with libzmq, CZMQ, pyzmq, etc. over **tcp** and **ipc**. The `inproc://` transport is OMQ-internal (in-process Ruby queues) and is not visible to native ZMQ running in the same process — use `ipc://` to talk across library boundaries.
204
+ See [CLI.md](CLI.md) for full documentation, or `omq --help` / `omq --examples`.
210
205
 
211
206
  ## Development
212
207
 
@@ -28,9 +28,11 @@ module OMQ
28
28
  sleep(config.delay) if config.delay && config.recv_only?
29
29
  wait_for_peer if needs_peer_wait?
30
30
 
31
- @sock.instance_exec(&@begin_proc) if @begin_proc
31
+ @sock.instance_exec(&@send_begin_proc) if @send_begin_proc
32
+ @sock.instance_exec(&@recv_begin_proc) if @recv_begin_proc
32
33
  run_loop(task)
33
- @sock.instance_exec(&@end_proc) if @end_proc
34
+ @sock.instance_exec(&@send_end_proc) if @send_end_proc
35
+ @sock.instance_exec(&@recv_end_proc) if @recv_end_proc
34
36
  ensure
35
37
  @sock&.close
36
38
  end
@@ -201,19 +203,19 @@ module OMQ
201
203
  end
202
204
  end
203
205
  elsif config.data || config.file
204
- parts = eval_expr(read_next)
206
+ parts = eval_send_expr(read_next)
205
207
  send_msg(parts) if parts
206
208
  elsif stdin_ready?
207
209
  loop do
208
210
  parts = read_next
209
211
  break unless parts
210
- parts = eval_expr(parts)
212
+ parts = eval_send_expr(parts)
211
213
  send_msg(parts) if parts
212
214
  i += 1
213
215
  break if n && n > 0 && i >= n
214
216
  end
215
- elsif @eval_proc
216
- parts = eval_expr(nil)
217
+ elsif @send_eval_proc
218
+ parts = eval_send_expr(nil)
217
219
  send_msg(parts) if parts
218
220
  end
219
221
  end
@@ -221,11 +223,11 @@ module OMQ
221
223
 
222
224
  def send_tick
223
225
  raw = read_next_or_nil
224
- if raw.nil? && !@eval_proc
226
+ if raw.nil? && !@send_eval_proc
225
227
  @send_tick_eof = true
226
228
  return 0
227
229
  end
228
- parts = eval_expr(raw)
230
+ parts = eval_send_expr(raw)
229
231
  send_msg(parts) if parts
230
232
  1
231
233
  end
@@ -237,7 +239,7 @@ module OMQ
237
239
  loop do
238
240
  parts = recv_msg
239
241
  break if parts.nil?
240
- parts = eval_expr(parts)
242
+ parts = eval_recv_expr(parts)
241
243
  output(parts)
242
244
  i += 1
243
245
  break if n && n > 0 && i >= n
@@ -246,7 +248,7 @@ module OMQ
246
248
 
247
249
 
248
250
  def wait_for_loops(receiver, sender)
249
- if config.data || config.file || config.expr || config.target
251
+ if config.data || config.file || config.send_expr || config.recv_expr || config.target
250
252
  sender.wait
251
253
  receiver.stop
252
254
  elsif config.count && config.count > 0
@@ -281,7 +283,8 @@ module OMQ
281
283
 
282
284
 
283
285
  def recv_msg_raw
284
- @sock.receive
286
+ msg = @sock.receive
287
+ msg&.dup
285
288
  end
286
289
 
287
290
 
@@ -320,7 +323,7 @@ module OMQ
320
323
  def read_next_or_nil
321
324
  if config.data || config.file
322
325
  read_next
323
- elsif @eval_proc
326
+ elsif @send_eval_proc
324
327
  nil
325
328
  else
326
329
  read_next
@@ -358,11 +361,30 @@ module OMQ
358
361
 
359
362
 
360
363
  def compile_expr
361
- return unless config.expr
362
- expr, begin_body, end_body = extract_blocks(config.expr)
363
- @begin_proc = eval("proc { #{begin_body} }") if begin_body
364
- @end_proc = eval("proc { #{end_body} }") if end_body
365
- @eval_proc = eval("proc { $_ = $F&.first; #{expr} }") if expr && !expr.strip.empty?
364
+ compile_one_expr(:send, config.send_expr)
365
+ compile_one_expr(:recv, config.recv_expr)
366
+ @send_eval_proc ||= wrap_registered_proc(OMQ.outgoing_proc)
367
+ @recv_eval_proc ||= wrap_registered_proc(OMQ.incoming_proc)
368
+ end
369
+
370
+
371
+ def wrap_registered_proc(block)
372
+ return unless block
373
+ proc do |msg|
374
+ $_ = msg&.first
375
+ block.call(msg)
376
+ end
377
+ end
378
+
379
+
380
+ def compile_one_expr(direction, src)
381
+ return unless src
382
+ expr, begin_body, end_body = extract_blocks(src)
383
+ instance_variable_set(:"@#{direction}_begin_proc", eval("proc { #{begin_body} }")) if begin_body
384
+ instance_variable_set(:"@#{direction}_end_proc", eval("proc { #{end_body} }")) if end_body
385
+ if expr && !expr.strip.empty?
386
+ instance_variable_set(:"@#{direction}_eval_proc", eval("proc { $_ = $F&.first; #{expr} }"))
387
+ end
366
388
  end
367
389
 
368
390
 
@@ -398,10 +420,21 @@ module OMQ
398
420
 
399
421
  SENT = Object.new.freeze # sentinel: eval already sent the reply
400
422
 
401
- def eval_expr(parts)
402
- return parts unless @eval_proc
423
+ def eval_send_expr(parts)
424
+ return parts unless @send_eval_proc
425
+ run_eval(@send_eval_proc, parts)
426
+ end
427
+
428
+
429
+ def eval_recv_expr(parts)
430
+ return parts unless @recv_eval_proc
431
+ run_eval(@recv_eval_proc, parts)
432
+ end
433
+
434
+
435
+ def run_eval(eval_proc, parts)
403
436
  $F = parts
404
- result = @sock.instance_exec(&@eval_proc)
437
+ result = @sock.instance_exec(parts, &eval_proc)
405
438
  return nil if result.nil?
406
439
  return SENT if result.equal?(@sock)
407
440
  return [result] if config.format == :marshal
@@ -411,7 +444,7 @@ module OMQ
411
444
  else [result.to_str]
412
445
  end
413
446
  rescue => e
414
- $stderr.puts "omq: -e error: #{e.message} (#{e.class})"
447
+ $stderr.puts "omq: eval error: #{e.message} (#{e.class})"
415
448
  exit 3
416
449
  end
417
450
 
@@ -11,7 +11,7 @@ module OMQ
11
11
 
12
12
 
13
13
  def run_loop(task)
14
- if config.echo || config.expr || config.data || config.file || !config.stdin_is_tty
14
+ if config.echo || config.recv_expr || @recv_eval_proc || config.data || config.file || !config.stdin_is_tty
15
15
  reply_loop
16
16
  else
17
17
  monitor_loop(task)
@@ -28,8 +28,8 @@ module OMQ
28
28
  routing_id = parts.shift
29
29
  body = @fmt.decompress(parts)
30
30
 
31
- if config.expr
32
- reply = eval_expr(body)
31
+ if config.recv_expr || @recv_eval_proc
32
+ reply = eval_recv_expr(body)
33
33
  output([display_routing_id(routing_id), *(reply || [""])])
34
34
  @sock.send_to(routing_id, @fmt.compress(reply || [""]).first)
35
35
  elsif config.echo
@@ -56,7 +56,7 @@ module OMQ
56
56
  break if parts.nil?
57
57
  routing_id = parts.shift
58
58
  parts = @fmt.decompress(parts)
59
- result = eval_expr([display_routing_id(routing_id), *parts])
59
+ result = eval_recv_expr([display_routing_id(routing_id), *parts])
60
60
  output(result)
61
61
  i += 1
62
62
  break if n && n > 0 && i >= n
@@ -71,18 +71,18 @@ module OMQ
71
71
  Async::Loop.quantized(interval: config.interval) do
72
72
  parts = read_next
73
73
  break unless parts
74
- send_targeted_or_plain(parts)
74
+ send_targeted_or_eval(parts)
75
75
  i += 1
76
76
  break if n && n > 0 && i >= n
77
77
  end
78
78
  elsif config.data || config.file
79
79
  parts = read_next
80
- send_targeted_or_plain(parts) if parts
80
+ send_targeted_or_eval(parts) if parts
81
81
  else
82
82
  loop do
83
83
  parts = read_next
84
84
  break unless parts
85
- send_targeted_or_plain(parts)
85
+ send_targeted_or_eval(parts)
86
86
  i += 1
87
87
  break if n && n > 0 && i >= n
88
88
  end
@@ -93,8 +93,13 @@ module OMQ
93
93
  end
94
94
 
95
95
 
96
- def send_targeted_or_plain(parts)
97
- if config.target
96
+ def send_targeted_or_eval(parts)
97
+ if @send_eval_proc
98
+ parts = eval_send_expr(parts)
99
+ return unless parts
100
+ routing_id = resolve_target(parts.shift)
101
+ @sock.send_to(routing_id, @fmt.compress(parts).first || "")
102
+ elsif config.target
98
103
  parts = @fmt.compress(parts)
99
104
  @sock.send_to(resolve_target(config.target), parts.first || "")
100
105
  else
@@ -2,6 +2,9 @@
2
2
 
3
3
  module OMQ
4
4
  module CLI
5
+ SEND_ONLY = %w[pub push scatter radio].freeze
6
+ RECV_ONLY = %w[sub pull gather dish].freeze
7
+
5
8
  Endpoint = Data.define(:url, :bind?) do
6
9
  def connect? = !bind?
7
10
  end
@@ -12,6 +15,8 @@ module OMQ
12
15
  :endpoints,
13
16
  :connects,
14
17
  :binds,
18
+ :in_endpoints,
19
+ :out_endpoints,
15
20
  :data,
16
21
  :file,
17
22
  :format,
@@ -29,7 +34,8 @@ module OMQ
29
34
  :heartbeat_ivl,
30
35
  :conflate,
31
36
  :compress,
32
- :expr,
37
+ :send_expr,
38
+ :recv_expr,
33
39
  :parallel,
34
40
  :transient,
35
41
  :verbose,
@@ -41,9 +47,6 @@ module OMQ
41
47
  :has_zstd,
42
48
  :stdin_is_tty,
43
49
  ) do
44
- SEND_ONLY = %w[pub push scatter radio].freeze
45
- RECV_ONLY = %w[sub pull gather dish].freeze
46
-
47
50
  def send_only? = SEND_ONLY.include?(type_name)
48
51
  def recv_only? = RECV_ONLY.include?(type_name)
49
52
  end
data/lib/omq/cli/pair.rb CHANGED
@@ -13,7 +13,7 @@ module OMQ
13
13
  loop do
14
14
  parts = recv_msg
15
15
  break if parts.nil?
16
- parts = eval_expr(parts)
16
+ parts = eval_recv_expr(parts)
17
17
  output(parts)
18
18
  i += 1
19
19
  break if n && n > 0 && i >= n
data/lib/omq/cli/pipe.rb CHANGED
@@ -24,9 +24,22 @@ module OMQ
24
24
  private
25
25
 
26
26
 
27
+ def resolve_endpoints
28
+ if config.in_endpoints.any?
29
+ [config.in_endpoints, config.out_endpoints]
30
+ else
31
+ [[config.endpoints[0]], [config.endpoints[1]]]
32
+ end
33
+ end
34
+
35
+
36
+ def attach_endpoints(sock, endpoints)
37
+ endpoints.each { |ep| ep.bind? ? sock.bind(ep.url) : sock.connect(ep.url) }
38
+ end
39
+
40
+
27
41
  def run_sequential(task)
28
- pull_ep = config.endpoints[0]
29
- push_ep = config.endpoints[1]
42
+ in_eps, out_eps = resolve_endpoints
30
43
 
31
44
  @pull = OMQ::PULL.new(linger: config.linger, recv_timeout: config.timeout)
32
45
  @push = OMQ::PUSH.new(linger: config.linger, send_timeout: config.timeout)
@@ -35,11 +48,11 @@ module OMQ
35
48
  @pull.heartbeat_interval = config.heartbeat_ivl if config.heartbeat_ivl
36
49
  @push.heartbeat_interval = config.heartbeat_ivl if config.heartbeat_ivl
37
50
 
38
- pull_ep.bind? ? @pull.bind(pull_ep.url) : @pull.connect(pull_ep.url)
39
- push_ep.bind? ? @push.bind(push_ep.url) : @push.connect(push_ep.url)
51
+ attach_endpoints(@pull, in_eps)
52
+ attach_endpoints(@push, out_eps)
40
53
 
41
54
  compile_expr
42
- @sock = @pull # for eval_expr instance_exec
55
+ @sock = @pull # for eval instance_exec
43
56
 
44
57
  with_timeout(config.timeout) do
45
58
  @push.peer_connected.wait
@@ -54,7 +67,7 @@ module OMQ
54
67
  end
55
68
  end
56
69
 
57
- @sock.instance_exec(&@begin_proc) if @begin_proc
70
+ @sock.instance_exec(&@recv_begin_proc) if @recv_begin_proc
58
71
 
59
72
  n = config.count
60
73
  i = 0
@@ -62,7 +75,7 @@ module OMQ
62
75
  parts = @pull.receive
63
76
  break if parts.nil?
64
77
  parts = @fmt.decompress(parts)
65
- parts = eval_expr(parts)
78
+ parts = eval_recv_expr(parts)
66
79
  if parts && !parts.empty?
67
80
  @push.send(@fmt.compress(parts))
68
81
  end
@@ -70,7 +83,7 @@ module OMQ
70
83
  break if n && n > 0 && i >= n
71
84
  end
72
85
 
73
- @sock.instance_exec(&@end_proc) if @end_proc
86
+ @sock.instance_exec(&@recv_end_proc) if @recv_end_proc
74
87
  ensure
75
88
  @pull&.close
76
89
  @push&.close
@@ -86,7 +99,7 @@ module OMQ
86
99
  Sync do |task|
87
100
  # Parse BEGIN/END blocks and per-message expression
88
101
  begin_proc = end_proc = eval_proc = nil
89
- if cfg.expr
102
+ if cfg.recv_expr
90
103
  extract = ->(src, kw) {
91
104
  s = src.index(/#{kw}\s*\{/)
92
105
  return [src, nil] unless s
@@ -97,7 +110,7 @@ module OMQ
97
110
  end
98
111
  [src[0...s] + src[j..], src[(i + 1)..(j - 2)]]
99
112
  }
100
- expr, begin_body = extract.(cfg.expr, "BEGIN")
113
+ expr, begin_body = extract.(cfg.recv_expr, "BEGIN")
101
114
  expr, end_body = extract.(expr, "END")
102
115
  begin_proc = eval("proc { #{begin_body} }") if begin_body
103
116
  end_proc = eval("proc { #{end_body} }") if end_body
@@ -115,8 +128,10 @@ module OMQ
115
128
  push.reconnect_interval = cfg.reconnect_ivl if cfg.reconnect_ivl
116
129
  pull.heartbeat_interval = cfg.heartbeat_ivl if cfg.heartbeat_ivl
117
130
  push.heartbeat_interval = cfg.heartbeat_ivl if cfg.heartbeat_ivl
118
- pull.connect(cfg.endpoints[0].url)
119
- push.connect(cfg.endpoints[1].url)
131
+ in_eps = cfg.in_endpoints.any? ? cfg.in_endpoints : [cfg.endpoints[0]]
132
+ out_eps = cfg.out_endpoints.any? ? cfg.out_endpoints : [cfg.endpoints[1]]
133
+ in_eps.each { |ep| pull.connect(ep.url) }
134
+ out_eps.each { |ep| push.connect(ep.url) }
120
135
 
121
136
  if cfg.timeout
122
137
  task.with_timeout(cfg.timeout) do
@@ -187,11 +202,12 @@ module OMQ
187
202
 
188
203
 
189
204
  def compile_expr
190
- return unless config.expr
191
- expr, begin_body, end_body = extract_blocks(config.expr)
192
- @begin_proc = eval("proc { #{begin_body} }") if begin_body
193
- @end_proc = eval("proc { #{end_body} }") if end_body
194
- @eval_proc = eval("proc { $_ = $F&.first; #{expr} }") if expr && !expr.strip.empty?
205
+ src = config.recv_expr
206
+ return unless src
207
+ expr, begin_body, end_body = extract_blocks(src)
208
+ @recv_begin_proc = eval("proc { #{begin_body} }") if begin_body
209
+ @recv_end_proc = eval("proc { #{end_body} }") if end_body
210
+ @recv_eval_proc = eval("proc { $_ = $F&.first; #{expr} }") if expr && !expr.strip.empty?
195
211
  end
196
212
 
197
213
 
@@ -224,10 +240,10 @@ module OMQ
224
240
  end
225
241
 
226
242
 
227
- def eval_expr(parts)
228
- return parts unless @eval_proc
243
+ def eval_recv_expr(parts)
244
+ return parts unless @recv_eval_proc
229
245
  $F = parts
230
- result = @sock.instance_exec(&@eval_proc)
246
+ result = @sock.instance_exec(&@recv_eval_proc)
231
247
  return nil if result.nil? || result.equal?(@sock)
232
248
  return [result] if config.format == :marshal
233
249
  case result
@@ -236,7 +252,7 @@ module OMQ
236
252
  else [result.to_str]
237
253
  end
238
254
  rescue => e
239
- $stderr.puts "omq: -e error: #{e.message} (#{e.class})"
255
+ $stderr.puts "omq: eval error: #{e.message} (#{e.class})"
240
256
  exit 3
241
257
  end
242
258
 
@@ -14,10 +14,12 @@ module OMQ
14
14
  loop do
15
15
  parts = read_next
16
16
  break unless parts
17
+ parts = eval_send_expr(parts)
18
+ next unless parts
17
19
  send_msg(parts)
18
20
  reply = recv_msg
19
21
  break if reply.nil?
20
- reply = eval_expr(reply)
22
+ reply = eval_recv_expr(reply)
21
23
  output(reply)
22
24
  i += 1
23
25
  break if n && n > 0 && i >= n
@@ -29,10 +31,12 @@ module OMQ
29
31
  loop do
30
32
  parts = read_next
31
33
  break unless parts
34
+ parts = eval_send_expr(parts)
35
+ next unless parts
32
36
  send_msg(parts)
33
37
  reply = recv_msg
34
38
  break if reply.nil?
35
- reply = eval_expr(reply)
39
+ reply = eval_recv_expr(reply)
36
40
  output(reply)
37
41
  i += 1
38
42
  break if n && n > 0 && i >= n
@@ -53,8 +57,8 @@ module OMQ
53
57
  loop do
54
58
  msg = recv_msg
55
59
  break if msg.nil?
56
- if config.expr
57
- reply = eval_expr(msg)
60
+ if config.recv_expr || @recv_eval_proc
61
+ reply = eval_recv_expr(msg)
58
62
  unless reply.equal?(SENT)
59
63
  output(reply)
60
64
  send_msg(reply || [""])
@@ -20,7 +20,7 @@ module OMQ
20
20
  identity = parts.shift
21
21
  parts.shift if parts.first == ""
22
22
  parts = @fmt.decompress(parts)
23
- result = eval_expr([display_routing_id(identity), *parts])
23
+ result = eval_recv_expr([display_routing_id(identity), *parts])
24
24
  output(result)
25
25
  i += 1
26
26
  break if n && n > 0 && i >= n
@@ -35,18 +35,18 @@ module OMQ
35
35
  Async::Loop.quantized(interval: config.interval) do
36
36
  parts = read_next
37
37
  break unless parts
38
- send_targeted_or_plain(parts)
38
+ send_targeted_or_eval(parts)
39
39
  i += 1
40
40
  break if n && n > 0 && i >= n
41
41
  end
42
42
  elsif config.data || config.file
43
43
  parts = read_next
44
- send_targeted_or_plain(parts) if parts
44
+ send_targeted_or_eval(parts) if parts
45
45
  else
46
46
  loop do
47
47
  parts = read_next
48
48
  break unless parts
49
- send_targeted_or_plain(parts)
49
+ send_targeted_or_eval(parts)
50
50
  i += 1
51
51
  break if n && n > 0 && i >= n
52
52
  end
@@ -57,8 +57,14 @@ module OMQ
57
57
  end
58
58
 
59
59
 
60
- def send_targeted_or_plain(parts)
61
- if config.target
60
+ def send_targeted_or_eval(parts)
61
+ if @send_eval_proc
62
+ parts = eval_send_expr(parts)
63
+ return unless parts
64
+ identity = resolve_target(parts.shift)
65
+ payload = @fmt.compress(parts)
66
+ @sock.send([identity, "", *payload])
67
+ elsif config.target
62
68
  payload = @fmt.compress(parts)
63
69
  @sock.send([resolve_target(config.target), "", *payload])
64
70
  else
data/lib/omq/cli.rb CHANGED
@@ -17,6 +17,15 @@ require_relative "cli/peer"
17
17
  require_relative "cli/pipe"
18
18
 
19
19
  module OMQ
20
+
21
+ class << self
22
+ attr_reader :outgoing_proc, :incoming_proc
23
+
24
+ def outgoing(&block) = @outgoing_proc = block
25
+ def incoming(&block) = @incoming_proc = block
26
+ end
27
+
28
+
20
29
  module CLI
21
30
  SOCKET_TYPE_NAMES = %w[
22
31
  req rep pub sub push pull pair dealer router
@@ -56,7 +65,7 @@ module OMQ
56
65
  └─────┘ "HELLO" └─────┘
57
66
 
58
67
  # terminal 1: echo server
59
- omq rep --bind tcp://:5555 --eval '$F.map(&:upcase)'
68
+ omq rep --bind tcp://:5555 --recv-eval '$F.map(&:upcase)'
60
69
 
61
70
  # terminal 2: send a request
62
71
  echo "hello" | omq req --connect tcp://localhost:5555
@@ -79,7 +88,7 @@ module OMQ
79
88
 
80
89
  ── Periodic Publish ───────────────────────────────────────────
81
90
 
82
- ┌─────┐ "tick 1" ┌─────┐
91
+ ┌─────┐ "tick 1" ┌─────┐
83
92
  │ PUB │──(every 1s)─→│ SUB │
84
93
  └─────┘ └─────┘
85
94
 
@@ -121,27 +130,31 @@ module OMQ
121
130
 
122
131
  # terminal 2: worker — uppercase each message
123
132
  omq pipe -c ipc://@work -c ipc://@sink -e '$F.map(&:upcase)'
124
-
125
133
  # terminal 3: collector
126
134
  omq pull --bind ipc://@sink
127
135
 
128
136
  # 4 Ractor workers in a single process (-P)
129
- omq pipe -c ipc://@work -c ipc://@sink -P 4 \
130
- -r ./fib.rb -e 'fib(Integer($_)).to_s'
137
+ omq pipe -c ipc://@work -c ipc://@sink -P4 -r./fib -e 'fib(Integer($_)).to_s'
131
138
 
132
139
  # exit when producer disconnects (--transient)
133
- omq pipe -c ipc://@work -c ipc://@sink --transient \
134
- -e '$F.map(&:upcase)'
140
+ omq pipe -c ipc://@work -c ipc://@sink --transient -e '$F.map(&:upcase)'
141
+
142
+ # fan-in: multiple sources → one sink
143
+ omq pipe --in -c ipc://@work1 -c ipc://@work2 \
144
+ --out -c ipc://@sink -e '$F.map(&:upcase)'
145
+
146
+ # fan-out: one source → multiple sinks (round-robin)
147
+ omq pipe --in -b tcp://:5555 --out -c ipc://@sink1 -c ipc://@sink2 -e '$F'
135
148
 
136
149
  ── CLIENT / SERVER (draft) ──────────────────────────────────
137
150
 
138
151
  ┌────────┐ "hello" ┌────────┐
139
- │ CLIENT │───────────→│ SERVER │ --eval '$F.map(&:upcase)'
152
+ │ CLIENT │───────────→│ SERVER │ --recv-eval '$F.map(&:upcase)'
140
153
  │ │←───────────│ │
141
154
  └────────┘ "HELLO" └────────┘
142
155
 
143
156
  # terminal 1: upcasing server
144
- omq server --bind tcp://:5555 --eval '$F.map(&:upcase)'
157
+ omq server --bind tcp://:5555 --recv-eval '$F.map(&:upcase)'
145
158
 
146
159
  # terminal 2: client
147
160
  echo "hello" | omq client --connect tcp://localhost:5555
@@ -187,30 +200,45 @@ module OMQ
187
200
  omq router --bind tcp://:5555
188
201
 
189
202
  # terminal 2: dealer with identity
190
- echo "hello" | omq dealer --connect tcp://localhost:5555 \
191
- --identity worker-1
203
+ echo "hello" | omq dealer --connect tcp://localhost:5555 --identity worker-1
192
204
 
193
205
  ── Ruby Eval ────────────────────────────────────────────────
194
206
 
195
- # filter: only pass messages containing "error"
196
- omq pull --bind tcp://:5557 \
197
- --eval '$F.first.include?("error") ? $F : nil'
207
+ # filter incoming: only pass messages containing "error"
208
+ omq pull -b tcp://:5557 --recv-eval '$F.first.include?("error") ? $F : nil'
198
209
 
199
- # transform with gems
200
- omq sub --connect tcp://localhost:5556 --require json \
201
- --eval 'JSON.parse($F.first)["temperature"]'
210
+ # transform incoming with gems
211
+ omq sub -c tcp://localhost:5556 -rjson -e 'JSON.parse($F.first)["temperature"]'
202
212
 
203
- # require a local file, use its methods in --eval
204
- omq rep --bind tcp://:5555 --require ./transform.rb \
205
- --eval 'upcase_all($F)'
213
+ # require a local file, use its methods
214
+ omq rep --bind tcp://:5555 --require ./transform.rb -e 'upcase_all($F)'
206
215
 
207
216
  # next skips, break stops — regexps match against $_
208
- omq pull --bind tcp://:5557 \
209
- --eval 'next if /^#/; break if /quit/; $F'
217
+ omq pull -b tcp://:5557 -e 'next if /^#/; break if /quit/; $F'
210
218
 
211
219
  # BEGIN/END blocks (like awk) — accumulate and summarize
212
- omq pull --bind tcp://:5557 \
213
- --eval 'BEGIN{ @sum = 0 } @sum += Integer($_); next END{ puts @sum }'
220
+ omq pull -b tcp://:5557 -e 'BEGIN{@sum = 0} @sum += Integer($_); nil END{puts @sum}'
221
+
222
+ # transform outgoing messages
223
+ echo hello | omq push -c tcp://localhost:5557 --send-eval '$F.map(&:upcase)'
224
+
225
+ # REQ: transform request and reply independently
226
+ echo hello | omq req -c tcp://localhost:5555 -E '$F.map(&:upcase)' -e '$_'
227
+
228
+ ── Script Handlers (-r) ────────────────────────────────────
229
+
230
+ # handler.rb — register transforms from a file
231
+ # db = PG.connect("dbname=app")
232
+ # OMQ.incoming { |first_part, _| db.exec(first_part).values.flatten }
233
+ # at_exit { db.close }
234
+ omq pull --bind tcp://:5557 -r./handler.rb
235
+
236
+ # combine script handlers with inline eval
237
+ omq req -c tcp://localhost:5555 -r./handler.rb -E '$F.map(&:upcase)'
238
+
239
+ # OMQ.outgoing { |msg| ... } — registered outgoing transform
240
+ # OMQ.incoming { |msg| ... } — registered incoming transform
241
+ # CLI flags (-e/-E) override registered handlers
214
242
  TEXT
215
243
 
216
244
  module_function
@@ -299,6 +327,8 @@ module OMQ
299
327
  endpoints: [],
300
328
  connects: [],
301
329
  binds: [],
330
+ in_endpoints: [],
331
+ out_endpoints: [],
302
332
  data: nil,
303
333
  file: nil,
304
334
  format: :ascii,
@@ -316,7 +346,8 @@ module OMQ
316
346
  heartbeat_ivl: nil,
317
347
  conflate: false,
318
348
  compress: false,
319
- expr: nil,
349
+ send_expr: nil,
350
+ recv_expr: nil,
320
351
  parallel: nil,
321
352
  transient: false,
322
353
  verbose: false,
@@ -326,6 +357,8 @@ module OMQ
326
357
  curve_server_key: nil,
327
358
  }
328
359
 
360
+ pipe_side = nil # nil = legacy positional mode; :in/:out = modal
361
+
329
362
  parser = OptionParser.new do |o|
330
363
  o.banner = "Usage: omq TYPE [options]\n\n" \
331
364
  "Types: req, rep, pub, sub, push, pull, pair, dealer, router\n" \
@@ -333,8 +366,24 @@ module OMQ
333
366
  "Virtual: pipe (PULL → eval → PUSH)\n\n"
334
367
 
335
368
  o.separator "Connection:"
336
- o.on("-c", "--connect URL", "Connect to endpoint (repeatable)") { |v| opts[:endpoints] << Endpoint.new(v, false); opts[:connects] << v }
337
- o.on("-b", "--bind URL", "Bind to endpoint (repeatable)") { |v| opts[:endpoints] << Endpoint.new(v, true); opts[:binds] << v }
369
+ o.on("-c", "--connect URL", "Connect to endpoint (repeatable)") { |v|
370
+ ep = Endpoint.new(v, false)
371
+ case pipe_side
372
+ when :in then opts[:in_endpoints] << ep
373
+ when :out then opts[:out_endpoints] << ep
374
+ else opts[:endpoints] << ep; opts[:connects] << v
375
+ end
376
+ }
377
+ o.on("-b", "--bind URL", "Bind to endpoint (repeatable)") { |v|
378
+ ep = Endpoint.new(v, true)
379
+ case pipe_side
380
+ when :in then opts[:in_endpoints] << ep
381
+ when :out then opts[:out_endpoints] << ep
382
+ else opts[:endpoints] << ep; opts[:binds] << v
383
+ end
384
+ }
385
+ o.on("--in", "Pipe: subsequent -b/-c attach to input (PULL) side") { pipe_side = :in }
386
+ o.on("--out", "Pipe: subsequent -b/-c attach to output (PUSH) side") { pipe_side = :out }
338
387
 
339
388
  o.separator "\nData source (REP: reply source):"
340
389
  o.on( "--echo", "Echo received messages back (REP)") { opts[:echo] = true }
@@ -380,9 +429,11 @@ module OMQ
380
429
  o.separator "\nCompression:"
381
430
  o.on("-z", "--compress", "Zstandard compression per frame") { opts[:compress] = true }
382
431
 
383
- o.separator "\nProcessing:"
384
- o.on("-e", "--eval EXPR", "Eval Ruby for each message ($F = parts)") { |v| opts[:expr] = v }
385
- o.on("-r", "--require LIB", "Require library or file (-r./lib.rb)") { |v|
432
+ o.separator "\nProcessing (-e = incoming, -E = outgoing):"
433
+ o.on("-e", "--recv-eval EXPR", "Eval Ruby for each incoming message ($F = parts)") { |v| opts[:recv_expr] = v }
434
+ o.on("-E", "--send-eval EXPR", "Eval Ruby for each outgoing message ($F = parts)") { |v| opts[:send_expr] = v }
435
+ o.on("-r", "--require LIB", "Require lib/file; scripts can register OMQ.outgoing/incoming") { |v|
436
+ require_relative "../omq" unless defined?(OMQ::VERSION)
386
437
  v.start_with?("./", "../") ? require(File.expand_path(v)) : require(v)
387
438
  }
388
439
  o.on("-P", "--parallel [N]", Integer, "Parallel Ractor workers (pipe only, default: nproc)") { |v|
@@ -421,10 +472,13 @@ module OMQ
421
472
 
422
473
  opts[:type_name] = type_name.downcase
423
474
 
424
- normalize = ->(url) { url.sub(%r{\Atcp://\*:}, "tcp://0.0.0.0:").sub(%r{\Atcp://:}, "tcp://localhost:") }
475
+ normalize = ->(url) { url.sub(%r{\Atcp://\*:}, "tcp://0.0.0.0:").sub(%r{\Atcp://:}, "tcp://localhost:") }
476
+ normalize_ep = ->(ep) { Endpoint.new(normalize.call(ep.url), ep.bind?) }
425
477
  opts[:binds].map!(&normalize)
426
478
  opts[:connects].map!(&normalize)
427
- opts[:endpoints].map! { |ep| Endpoint.new(normalize.call(ep.url), ep.bind?) }
479
+ opts[:endpoints].map!(&normalize_ep)
480
+ opts[:in_endpoints].map!(&normalize_ep)
481
+ opts[:out_endpoints].map!(&normalize_ep)
428
482
 
429
483
  opts
430
484
  end
@@ -436,8 +490,16 @@ module OMQ
436
490
  type_name = opts[:type_name]
437
491
 
438
492
  if type_name == "pipe"
439
- abort "pipe requires exactly 2 endpoints (pull-side and push-side)" if opts[:endpoints].size != 2
493
+ has_in_out = opts[:in_endpoints].any? || opts[:out_endpoints].any?
494
+ if has_in_out
495
+ abort "pipe --in requires at least one endpoint" if opts[:in_endpoints].empty?
496
+ abort "pipe --out requires at least one endpoint" if opts[:out_endpoints].empty?
497
+ abort "pipe: don't mix --in/--out with bare -b/-c endpoints" unless opts[:endpoints].empty?
498
+ else
499
+ abort "pipe requires exactly 2 endpoints (pull-side and push-side), or use --in/--out" if opts[:endpoints].size != 2
500
+ end
440
501
  else
502
+ abort "--in/--out are only valid for pipe" if opts[:in_endpoints].any? || opts[:out_endpoints].any?
441
503
  abort "At least one --connect or --bind is required" if opts[:connects].empty? && opts[:binds].empty?
442
504
  end
443
505
  abort "--data and --file are mutually exclusive" if opts[:data] && opts[:file]
@@ -447,11 +509,15 @@ module OMQ
447
509
  abort "--identity is only valid for DEALER/ROUTER" if opts[:identity] && !%w[dealer router].include?(type_name)
448
510
  abort "--target is only valid for ROUTER/SERVER/PEER" if opts[:target] && !%w[router server peer].include?(type_name)
449
511
  abort "--conflate is only valid for PUB/RADIO" if opts[:conflate] && !%w[pub radio].include?(type_name)
512
+ abort "--recv-eval is not valid for send-only sockets (use --send-eval / -E)" if opts[:recv_expr] && SEND_ONLY.include?(type_name)
513
+ abort "--send-eval is not valid for recv-only sockets (use --recv-eval / -e)" if opts[:send_expr] && RECV_ONLY.include?(type_name)
514
+ abort "--send-eval and --target are mutually exclusive" if opts[:send_expr] && opts[:target]
450
515
 
451
516
  if opts[:parallel]
452
517
  abort "-P/--parallel is only valid for pipe" unless type_name == "pipe"
453
518
  abort "-P/--parallel must be >= 2" if opts[:parallel] < 2
454
- abort "-P/--parallel requires both endpoints to use --connect (not --bind)" if opts[:endpoints].any?(&:bind?)
519
+ all_pipe_eps = opts[:in_endpoints] + opts[:out_endpoints] + opts[:endpoints]
520
+ abort "-P/--parallel requires all endpoints to use --connect (not --bind)" if all_pipe_eps.any?(&:bind?)
455
521
  end
456
522
 
457
523
  (opts[:connects] + opts[:binds]).each do |url|
data/lib/omq/version.rb CHANGED
@@ -1,5 +1,5 @@
1
1
  # frozen_string_literal: true
2
2
 
3
3
  module OMQ
4
- VERSION = "0.6.5"
4
+ VERSION = "0.7.0"
5
5
  end
@@ -215,29 +215,50 @@ module OMQ
215
215
  # fast path when the connection is a DirectPipe.
216
216
  #
217
217
  # @param conn [Connection, Transport::Inproc::DirectPipe]
218
+ # Starts a recv pump that dequeues messages from a connection
219
+ # and enqueues them into the routing strategy's recv queue.
220
+ #
221
+ # When a block is given, each message is yielded for transformation
222
+ # before enqueueing. The block is compiled at the call site, giving
223
+ # YJIT a monomorphic call per routing strategy instead of a shared
224
+ # megamorphic `transform.call` dispatch.
225
+ #
226
+ # @param conn [Connection, Transport::Inproc::DirectPipe]
218
227
  # @param recv_queue [Async::LimitedQueue] routing strategy's recv queue
219
- # @param transform [#call, nil] optional message transform
228
+ # @yield [msg] optional per-message transform
220
229
  # @return [#stop, nil] pump task handle, or nil for DirectPipe bypass
221
230
  #
222
- def start_recv_pump(conn, recv_queue, transform: nil)
231
+ def start_recv_pump(conn, recv_queue, &transform)
223
232
  if conn.is_a?(Transport::Inproc::DirectPipe) && conn.peer
224
233
  conn.peer.direct_recv_queue = recv_queue
225
234
  conn.peer.direct_recv_transform = transform
226
235
  return nil
227
236
  end
228
237
 
229
- Reactor.spawn_pump(annotation: "recv pump") do
230
- loop do
231
- msg = conn.receive_message
232
- msg = transform ? transform.call(msg).freeze : msg
233
- recv_queue.enqueue(msg)
238
+ if transform
239
+ Reactor.spawn_pump(annotation: "recv pump") do
240
+ loop do
241
+ msg = conn.receive_message
242
+ msg = transform.call(msg).freeze
243
+ recv_queue.enqueue(msg)
244
+ end
245
+ rescue Async::Stop
246
+ rescue ProtocolError, *CONNECTION_LOST
247
+ connection_lost(conn)
248
+ rescue => error
249
+ signal_fatal_error(error)
250
+ end
251
+ else
252
+ Reactor.spawn_pump(annotation: "recv pump") do
253
+ loop do
254
+ recv_queue.enqueue(conn.receive_message)
255
+ end
256
+ rescue Async::Stop
257
+ rescue ProtocolError, *CONNECTION_LOST
258
+ connection_lost(conn)
259
+ rescue => error
260
+ signal_fatal_error(error)
234
261
  end
235
- rescue Async::Stop
236
- # normal shutdown
237
- rescue ProtocolError, *CONNECTION_LOST
238
- connection_lost(conn)
239
- rescue => error
240
- signal_fatal_error(error)
241
262
  end
242
263
  end
243
264
 
@@ -34,8 +34,9 @@ module OMQ
34
34
  routing_id = SecureRandom.bytes(4)
35
35
  @connections_by_routing_id[routing_id] = connection
36
36
 
37
- task = @engine.start_recv_pump(connection, @recv_queue,
38
- transform: ->(msg) { [routing_id, *msg] })
37
+ task = @engine.start_recv_pump(connection, @recv_queue) do |msg|
38
+ [routing_id, *msg]
39
+ end
39
40
  @tasks << task if task
40
41
 
41
42
  start_send_pump unless @send_pump_started
@@ -29,14 +29,13 @@ module OMQ
29
29
  # @param connection [Connection]
30
30
  #
31
31
  def connection_added(connection)
32
- transform = ->(msg) {
32
+ task = @engine.start_recv_pump(connection, @recv_queue) do |msg|
33
33
  delimiter = msg.index(&:empty?) || msg.size
34
34
  envelope = msg[0, delimiter]
35
35
  body = msg[(delimiter + 1)..] || []
36
36
  @pending_replies << { conn: connection, envelope: envelope }
37
37
  body
38
- }
39
- task = @engine.start_recv_pump(connection, @recv_queue, transform: transform)
38
+ end
40
39
  @tasks << task if task
41
40
  start_send_pump unless @send_pump_started
42
41
  end
@@ -28,8 +28,9 @@ module OMQ
28
28
  def connection_added(connection)
29
29
  @connections << connection
30
30
  signal_connection_available
31
- task = @engine.start_recv_pump(connection, @recv_queue,
32
- transform: method(:transform_recv))
31
+ task = @engine.start_recv_pump(connection, @recv_queue) do |msg|
32
+ msg.first&.empty? ? msg[1..] : msg
33
+ end
33
34
  @tasks << task if task
34
35
  start_send_pump unless @send_pump_started
35
36
  end
@@ -58,11 +59,6 @@ module OMQ
58
59
  #
59
60
  def transform_send(parts) = ["".b, *parts]
60
61
 
61
- # REQ strips the leading empty delimiter frame on receive.
62
- #
63
- def transform_recv(msg)
64
- msg.first&.empty? ? msg[1..] : msg
65
- end
66
62
  end
67
63
  end
68
64
  end
@@ -35,8 +35,9 @@ module OMQ
35
35
  identity = SecureRandom.bytes(5) if identity.nil? || identity.empty?
36
36
  @connections_by_identity[identity] = connection
37
37
 
38
- task = @engine.start_recv_pump(connection, @recv_queue,
39
- transform: ->(msg) { [identity, *msg] })
38
+ task = @engine.start_recv_pump(connection, @recv_queue) do |msg|
39
+ [identity, *msg]
40
+ end
40
41
  @tasks << task if task
41
42
 
42
43
  start_send_pump unless @send_pump_started
@@ -34,8 +34,9 @@ module OMQ
34
34
  routing_id = SecureRandom.bytes(4)
35
35
  @connections_by_routing_id[routing_id] = connection
36
36
 
37
- task = @engine.start_recv_pump(connection, @recv_queue,
38
- transform: ->(msg) { [routing_id, *msg] })
37
+ task = @engine.start_recv_pump(connection, @recv_queue) do |msg|
38
+ [routing_id, *msg]
39
+ end
39
40
  @tasks << task if task
40
41
 
41
42
  start_send_pump unless @send_pump_started
data/lib/omq.rb CHANGED
@@ -17,6 +17,7 @@ module OMQ
17
17
  #
18
18
  class SocketDeadError < RuntimeError; end
19
19
  end
20
+
20
21
  require_relative "omq/zmtp"
21
22
  require_relative "omq/socket"
22
23
  require_relative "omq/req_rep"
metadata CHANGED
@@ -1,7 +1,7 @@
1
1
  --- !ruby/object:Gem::Specification
2
2
  name: omq
3
3
  version: !ruby/object:Gem::Version
4
- version: 0.6.5
4
+ version: 0.7.0
5
5
  platform: ruby
6
6
  authors:
7
7
  - Patrik Wenger