omq 0.20.0 → 0.21.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: 6800afbb9c1e8dd9f7a973f7105ff94a13bf99bc4f6a5c302fc3ff7c2ee3220a
4
- data.tar.gz: a166c7b6d54596565574b24183d64c649b8a58c8d826bf3074eb800cb9f0f52f
3
+ metadata.gz: f24ac91fe456168b4d369f63506cf1d3d2b48488637ce767363aa17bb542d2b3
4
+ data.tar.gz: ca1b98ab4083ad90f7f483bc9df937656ae9862c9fd07800fc46315543e5fe75
5
5
  SHA512:
6
- metadata.gz: 50c173ee88a276291a82864adadeb11500265d3df24164de183407af0b1586498105b4a837a75027f2762b5a91a9f70ad6dbf0a718f9dc179e920ba3a238e266
7
- data.tar.gz: da22b101249f108c9dc59763f1589fb1cd350b73bdbd62a3f59424079db68cb892bc1a93052cab603ae0f62eefb35fee1ff3a10743f500ed68dab622d3a3a027
6
+ metadata.gz: 16f20fa600ebd66228589da4e03efc809a616d6a74fbe3460a8ced6811349fb86903e43a440708e1e78f102af711d6f9ea3ebf9cbc0e7fb4aef9591db5061e39
7
+ data.tar.gz: 89ff3da1bb5caa2223d1e323a389cacc4f270fed719902dbd0721c56da8ff5c0a1cbf1e6eebc59c8386f470c433c69dfb20462d46f84c9a2fa848b3e1364dfd0
data/CHANGELOG.md CHANGED
@@ -1,5 +1,37 @@
1
1
  # Changelog
2
2
 
3
+ ## 0.21.0 — 2026-04-15
4
+
5
+ ### Changed
6
+
7
+ - **Recv path: shared queue, no more `FairQueue`.** Every fair-queue
8
+ routing strategy (Pull, Pair, Rep, Dealer, Router, Req, Sub, XSub)
9
+ now owns a single `Async::LimitedQueue` sized to `recv_hwm`. Each
10
+ connection's recv pump writes directly into it. `FairQueue`,
11
+ `SignalingQueue`, and the `FairRecv` mixin are deleted. Cross-peer
12
+ fairness comes entirely from the pump yield limit; per-connection
13
+ ordering is preserved; cross-connection ordering was never a
14
+ guarantee. Symmetric with the send side, which already uses one
15
+ work-stealing queue per socket. Sister gems (channel, clientserver,
16
+ p2p, radiodish, scattergather, qos) updated to match.
17
+ - **Recv pump fairness bumped to 256 msgs / 512 KiB** (was 64 / 1 MiB),
18
+ symmetric with `RoundRobin::BATCH_MSG_CAP` / `BATCH_BYTE_CAP` on the
19
+ send side.
20
+
21
+ ### Added
22
+
23
+ - **`Socket#attach_endpoints` accepts arrays.** Constructors passed an
24
+ array of endpoint strings now bind/connect each one in order, so
25
+ `OMQ::SUB.new(["inproc://a", "inproc://b"])` works.
26
+ - **PUB/SUB regression test** for a SUB with sequential post-hoc
27
+ `#connect` calls to multiple bound PUBs, mirroring the SCATTER/GATHER
28
+ post-hoc-connect coverage.
29
+ - **DESIGN.md: "Libzmq quirks OMQ avoids"** — per-pipe HWM (actual
30
+ buffering is `send_hwm × N_peers`, forces strict RR, slow-worker
31
+ stall footgun) and the edge-triggered `ZMQ_FD` that fires spuriously
32
+ and misses edges, requiring the `ZMQ_EVENTS` / `ZMQ_DONTWAIT` dance.
33
+ README "Socket Types" condensed to a pointer at DESIGN.md.
34
+
3
35
  ## 0.20.0 — 2026-04-14
4
36
 
5
37
  ### Changed
data/README.md CHANGED
@@ -5,29 +5,45 @@
5
5
  [![License: ISC](https://img.shields.io/badge/License-ISC-blue.svg)](LICENSE)
6
6
  [![Ruby](https://img.shields.io/badge/Ruby-%3E%3D%203.3-CC342D?logo=ruby&logoColor=white)](https://www.ruby-lang.org)
7
7
 
8
- > **932k msg/s** inproc | **328k msg/s** ipc | **329k msg/s** tcp
8
+ > **1.23M msg/s** inproc | **361k msg/s** ipc | **358k msg/s** tcp
9
9
  >
10
- > **11.5 µs** inproc latency | **54 µs** ipc | **69 µs** tcp
10
+ > **9.1 µs** inproc latency | **49 µs** ipc | **64 µs** tcp
11
11
  >
12
12
  > Ruby 4.0 + YJIT on a Linux VM — see [`bench/`](bench/) for full results
13
13
 
14
- `gem install omq` and you're done. No libzmq, no compiler, no system packages — just Ruby talking to every other ZeroMQ peer out there.
14
+ `gem install omq` and you're done. No libzmq, no compiler, no system packages —
15
+ just Ruby talking to every other ZeroMQ peer out there.
15
16
 
16
- ØMQ gives your Ruby processes a way to talk to each other — and to anything else speaking ZeroMQ — without a broker in the middle. Same API whether they live in the same process, on the same machine, or across the network. Reconnects, queuing, and back-pressure are handled for you; you write the interesting part.
17
+ ØMQ gives your Ruby processes a way to talk to each other — and to anything
18
+ else speaking ZeroMQ — without a broker in the middle. Same API whether they
19
+ live in the same process, on the same machine, or across the network.
20
+ Reconnects, queuing, and back-pressure are handled for you; you write the
21
+ interesting part.
17
22
 
18
- New to ZeroMQ? Start with [GETTING_STARTED.md](GETTING_STARTED.md) — a ~30 min walkthrough of every major pattern with working code.
23
+ New to ZeroMQ? Start with [GETTING_STARTED.md](GETTING_STARTED.md) — a ~30 min
24
+ walkthrough of every major pattern with working code.
19
25
 
20
26
  ## Highlights
21
27
 
22
- - **Zero dependencies on C** — no extensions, no FFI, no libzmq. `gem install` just works everywhere
23
- - **Fast** — YJIT-optimized hot paths, batched sends, GC-tuned allocations, buffered I/O via [io-stream](https://github.com/socketry/io-stream), direct-pipe inproc bypass
24
- - **[`omq` CLI](https://github.com/paddor/omq-cli)** — a powerful swiss army knife for ØMQ. `gem install omq-cli`
25
- - **Every socket pattern** — req/rep, pub/sub, push/pull, dealer/router, xpub/xsub, pair, and all draft types
26
- - **Every transport** — tcp, ipc (Unix domain sockets), inproc (in-process queues)
28
+ - **Zero dependencies on C** — no extensions, no FFI, no libzmq. `gem install`
29
+ just works everywhere
30
+ - **Fast** — YJIT-optimized hot paths, batched sends, GC-tuned allocations,
31
+ buffered I/O via [io-stream](https://github.com/socketry/io-stream),
32
+ direct-pipe inproc bypass
33
+ - **[`omq` CLI](https://github.com/paddor/omq-cli)** — a powerful swiss army
34
+ knife for ØMQ. `gem install omq-cli`
35
+ - **Every socket pattern** — req/rep, pub/sub, push/pull, dealer/router,
36
+ xpub/xsub, pair, and all draft types
37
+ - **Every transport** — tcp, ipc (Unix domain sockets), inproc (in-process
38
+ queues)
27
39
  - **Async-native** — built on fibers, non-blocking from the ground up
28
- - **Works outside Async too** — a shared IO thread handles sockets for callers that aren't inside a reactor, so simple scripts just work
29
- - **Wire-compatible** interoperates with libzmq, pyzmq, CZMQ, zmq.rs over tcp and ipc
30
- - **Bind/connect order doesn't matter** — connect before bind, bind before connect, peers come and go. ZeroMQ reconnects automatically and queued messages drain when peers arrive
40
+ - **Works outside Async too** — a shared IO thread handles sockets for callers
41
+ that aren't inside a reactor, so simple scripts just work
42
+ - **Wire-compatible** — interoperates with libzmq, pyzmq, CZMQ, zmq.rs over tcp
43
+ and ipc
44
+ - **Bind/connect order doesn't matter** — connect before bind, bind before
45
+ connect, peers come and go. ZeroMQ reconnects automatically and queued
46
+ messages drain when peers arrive
31
47
 
32
48
  For architecture internals, see [DESIGN.md](DESIGN.md).
33
49
 
@@ -99,7 +115,8 @@ end
99
115
 
100
116
  ### Without Async (IO thread)
101
117
 
102
- OMQ spawns a shared `omq-io` thread when used outside an Async reactor — no boilerplate needed:
118
+ OMQ spawns a shared `omq-io` thread when used outside an Async reactor — no
119
+ boilerplate needed:
103
120
 
104
121
  ```ruby
105
122
  require 'omq'
@@ -114,7 +131,8 @@ push.close
114
131
  pull.close
115
132
  ```
116
133
 
117
- The IO thread runs all pumps, reconnection, and heartbeating in the background. When you're inside an `Async` block, OMQ uses the existing reactor instead.
134
+ The IO thread runs all pumps, reconnection, and heartbeating in the background.
135
+ When you're inside an `Async` block, OMQ uses the existing reactor instead.
118
136
 
119
137
  ### Queue Interface
120
138
 
@@ -138,7 +156,11 @@ end
138
156
 
139
157
  ## Socket Types
140
158
 
141
- All sockets are thread-safe. Default HWM is 1000 messages per socket. `max_message_size` defaults to **`nil` (unlimited)** — set `socket.max_message_size = N` to cap inbound frames at `N` bytes; oversized frames cause the connection to be dropped before the body is read from the wire. Classes live under `OMQ::` (alias: `ØMQ`).
159
+ All sockets are thread-safe. Default HWM is 1000 messages per socket.
160
+ `max_message_size` defaults to **`nil` (unlimited)** — set
161
+ `socket.max_message_size = N` to cap inbound frames at `N` bytes; oversized
162
+ frames cause the connection to be dropped before the body is read from the
163
+ wire. Classes live under `OMQ::` (alias: `ØMQ`).
142
164
 
143
165
  #### Standard (multipart messages)
144
166
 
@@ -151,7 +173,11 @@ All sockets are thread-safe. Default HWM is 1000 messages per socket. `max_messa
151
173
  | **XPUB** / **XSUB** | Fan-out (subscription events) | Fair-queue | Drop |
152
174
  | **PAIR** | Exclusive 1-to-1 | Exclusive 1-to-1 | Block |
153
175
 
154
- > **Work-stealing vs. round-robin.** libzmq uses strict per-pipe round-robin for outbound load balancing — message N goes to peer N mod K regardless of whether that peer is busy. OMQ uses **work-stealing**: one shared send queue per socket and N pump fibers that race to drain it. Whichever pump is ready next picks up the next batch, so a slow peer can't stall the pipeline. The trade-off: distribution is not strict round-robin under bursts. If a producer enqueues a large burst before any pump fiber gets scheduled, the first pump to wake will dequeue up to one whole batch (256 messages or 512 KB, whichever hits first) in a single non-blocking drain — so a tight `n.times { sock << msg }` loop on a small `n` may dump everything on one peer. Slow or steady producers don't see this: each pump dequeues one message, writes, re-parks, and the FIFO wait queue gives every pump a fair turn. Burst distribution also evens out once the burst exceeds one pump's batch cap. See [DESIGN.md](DESIGN.md#per-socket-hwm-not-per-connection) for the full reasoning.
176
+ > **Work-stealing, not round-robin.** Outbound load balancing uses one shared
177
+ > send queue per socket drained by N racing pump fibers, so a slow peer can't
178
+ > stall the pipeline. Under tight bursts on small `n`, distribution isn't
179
+ > strict RR. See [DESIGN.md](DESIGN.md#per-socket-hwm-not-per-connection) and
180
+ > [Libzmq quirks](DESIGN.md#libzmq-quirks-omq-avoids) for the reasoning.
155
181
 
156
182
  #### Draft (single-frame only)
157
183
 
@@ -167,7 +193,8 @@ Each draft pattern lives in its own gem — install only the ones you use.
167
193
 
168
194
  ## CLI
169
195
 
170
- Install [omq-cli](https://github.com/paddor/omq-cli) for a command-line tool that sends, receives, pipes, and transforms ZeroMQ messages from the terminal:
196
+ Install [omq-cli](https://github.com/paddor/omq-cli) for a command-line tool
197
+ that sends, receives, pipes, and transforms ZeroMQ messages from the terminal:
171
198
 
172
199
  ```sh
173
200
  gem install omq-cli
@@ -180,14 +207,21 @@ See the [omq-cli README](https://github.com/paddor/omq-cli) for full documentati
180
207
 
181
208
  ## Companion Gems
182
209
 
183
- - **[omq-ffi](https://github.com/paddor/omq-ffi)** — libzmq FFI backend. Same OMQ socket API, but backed by libzmq instead of the pure Ruby ZMTP stack. Useful for interop testing and when you need libzmq-specific features. Requires libzmq installed.
184
- - **[omq-ractor](https://github.com/paddor/omq-ractor)** — bridge OMQ sockets into Ruby Ractors for true parallel processing across cores. I/O stays on the main Ractor, worker Ractors do pure computation.
210
+ - **[omq-ffi](https://github.com/paddor/omq-ffi)** — libzmq FFI backend. Same
211
+ OMQ socket API, but backed by libzmq instead of the pure Ruby ZMTP stack.
212
+ Useful for interop testing and when you need libzmq-specific features.
213
+ Requires libzmq installed.
214
+ - **[omq-ractor](https://github.com/paddor/omq-ractor)** — bridge OMQ sockets
215
+ into Ruby Ractors for true parallel processing across cores. I/O stays on the
216
+ main Ractor, worker Ractors do pure computation.
185
217
 
186
218
  ### Protocol extensions (RFCs)
187
219
 
188
- Optional plug-ins that extend the ZMTP wire protocol. Each is a separate gem; load the ones you need.
220
+ Optional plug-ins that extend the ZMTP wire protocol. Each is a separate gem;
221
+ load the ones you need.
189
222
 
190
- - **[omq-rfc-zstd](https://github.com/paddor/omq-rfc-zstd)** — transparent Zstandard compression on the wire, negotiated per peer via READY properties.
223
+ - **[omq-rfc-zstd](https://github.com/paddor/omq-rfc-zstd)** — transparent
224
+ Zstandard compression on the wire, negotiated per peer via READY properties.
191
225
 
192
226
  ## Development
193
227
 
@@ -15,22 +15,22 @@ module OMQ
15
15
  class RecvPump
16
16
  # Max messages read from one connection before yielding to the
17
17
  # scheduler. Prevents a busy peer from starving its siblings in
18
- # fair-queue recv sockets.
19
- FAIRNESS_MESSAGES = 64
18
+ # fair-queue recv sockets. Symmetric with RoundRobin send batching.
19
+ FAIRNESS_MESSAGES = 256
20
20
 
21
21
 
22
22
  # Max bytes read from one connection before yielding. Only counted
23
23
  # for ZMTP connections (inproc skips the check). Complements
24
24
  # {FAIRNESS_MESSAGES}: small-message floods are bounded by count,
25
- # large-message floods by bytes.
26
- FAIRNESS_BYTES = 1 << 20 # 1 MB
25
+ # large-message floods by bytes. Symmetric with RoundRobin send batching.
26
+ FAIRNESS_BYTES = 512 * 1024
27
27
 
28
28
 
29
29
  # Public entry point — callers use the class method.
30
30
  #
31
31
  # @param parent [Async::Task, Async::Barrier] parent to spawn under
32
32
  # @param conn [Connection, Transport::Inproc::DirectPipe]
33
- # @param recv_queue [SignalingQueue]
33
+ # @param recv_queue [Async::LimitedQueue]
34
34
  # @param engine [Engine]
35
35
  # @param transform [Proc, nil]
36
36
  # @return [Async::Task, nil]
@@ -41,7 +41,7 @@ module OMQ
41
41
 
42
42
 
43
43
  # @param conn [Connection, Transport::Inproc::DirectPipe]
44
- # @param recv_queue [Routing::SignalingQueue]
44
+ # @param recv_queue [Async::LimitedQueue]
45
45
  # @param engine [Engine]
46
46
  #
47
47
  def initialize(conn, recv_queue, engine)
data/lib/omq/engine.rb CHANGED
@@ -273,7 +273,7 @@ module OMQ
273
273
  # Starts a recv pump for a connection, or wires the inproc fast path.
274
274
  #
275
275
  # @param conn [Connection, Transport::Inproc::DirectPipe]
276
- # @param recv_queue [SignalingQueue]
276
+ # @param recv_queue [Async::LimitedQueue]
277
277
  # @yield [msg] optional per-message transform
278
278
  # @return [Async::Task, nil]
279
279
  #
@@ -8,10 +8,9 @@ module OMQ
8
8
  #
9
9
  class Dealer
10
10
  include RoundRobin
11
- include FairRecv
12
11
 
13
12
 
14
- # @return [FairQueue]
13
+ # @return [Async::LimitedQueue]
15
14
  #
16
15
  attr_reader :recv_queue
17
16
 
@@ -20,16 +19,35 @@ module OMQ
20
19
  #
21
20
  def initialize(engine)
22
21
  @engine = engine
23
- @recv_queue = FairQueue.new
22
+ @recv_queue = Routing.build_queue(engine.options.recv_hwm, :block)
24
23
  @tasks = []
25
24
  init_round_robin(engine)
26
25
  end
27
26
 
28
27
 
28
+ # Dequeues the next received message. Blocks until one is available.
29
+ #
30
+ # @return [Array<String>, nil]
31
+ #
32
+ def dequeue_recv
33
+ @recv_queue.dequeue
34
+ end
35
+
36
+
37
+ # Wakes a blocked {#dequeue_recv} with a nil sentinel.
38
+ #
39
+ # @return [void]
40
+ #
41
+ def unblock_recv
42
+ @recv_queue.enqueue(nil)
43
+ end
44
+
45
+
29
46
  # @param connection [Connection]
30
47
  #
31
48
  def connection_added(connection)
32
- add_fair_recv_connection(connection)
49
+ task = @engine.start_recv_pump(connection, @recv_queue)
50
+ @tasks << task if task
33
51
  add_round_robin_send_connection(connection)
34
52
  end
35
53
 
@@ -38,7 +56,6 @@ module OMQ
38
56
  #
39
57
  def connection_removed(connection)
40
58
  @connections.delete(connection)
41
- @recv_queue.remove_queue(connection)
42
59
  remove_round_robin_send_connection(connection)
43
60
  end
44
61
 
@@ -10,10 +10,7 @@ module OMQ
10
10
  # the in-flight batch is dropped (matching libzmq).
11
11
  #
12
12
  class Pair
13
- include FairRecv
14
-
15
-
16
- # @return [FairQueue]
13
+ # @return [Async::LimitedQueue]
17
14
  #
18
15
  attr_reader :recv_queue
19
16
 
@@ -23,13 +20,31 @@ module OMQ
23
20
  def initialize(engine)
24
21
  @engine = engine
25
22
  @connection = nil
26
- @recv_queue = FairQueue.new
27
- @send_queue = Routing.build_queue(@engine.options.send_hwm, :block)
23
+ @recv_queue = Routing.build_queue(engine.options.recv_hwm, :block)
24
+ @send_queue = Routing.build_queue(engine.options.send_hwm, :block)
28
25
  @send_pump = nil
29
26
  @tasks = []
30
27
  end
31
28
 
32
29
 
30
+ # Dequeues the next received message. Blocks until one is available.
31
+ #
32
+ # @return [Array<String>, nil]
33
+ #
34
+ def dequeue_recv
35
+ @recv_queue.dequeue
36
+ end
37
+
38
+
39
+ # Wakes a blocked {#dequeue_recv} with a nil sentinel.
40
+ #
41
+ # @return [void]
42
+ #
43
+ def unblock_recv
44
+ @recv_queue.enqueue(nil)
45
+ end
46
+
47
+
33
48
  # @param connection [Connection]
34
49
  # @raise [RuntimeError] if a connection already exists
35
50
  #
@@ -37,7 +52,8 @@ module OMQ
37
52
  raise "PAIR allows only one peer" if @connection
38
53
  @connection = connection
39
54
 
40
- add_fair_recv_connection(connection)
55
+ task = @engine.start_recv_pump(connection, @recv_queue)
56
+ @tasks << task if task
41
57
 
42
58
  unless connection.is_a?(Transport::Inproc::DirectPipe)
43
59
  start_send_pump(connection)
@@ -50,7 +66,6 @@ module OMQ
50
66
  def connection_removed(connection)
51
67
  if @connection == connection
52
68
  @connection = nil
53
- @recv_queue.remove_queue(connection)
54
69
  @send_pump&.stop
55
70
  @send_pump = nil
56
71
  end
@@ -5,35 +5,52 @@ module OMQ
5
5
  # PULL socket routing: fair-queue receive from PUSH peers.
6
6
  #
7
7
  class Pull
8
- include FairRecv
9
-
10
-
11
8
  # @param engine [Engine]
12
9
  #
13
10
  def initialize(engine)
14
11
  @engine = engine
15
- @recv_queue = FairQueue.new
12
+ @recv_queue = Routing.build_queue(engine.options.recv_hwm, :block)
16
13
  @tasks = []
17
14
  end
18
15
 
19
16
 
20
- # @return [FairQueue]
17
+ # @return [Async::LimitedQueue]
21
18
  #
22
19
  attr_reader :recv_queue
23
20
 
24
21
 
22
+ # Dequeues the next received message. Blocks until one is available.
23
+ # Engine-facing contract — Engine must not touch @recv_queue directly.
24
+ #
25
+ # @return [Array<String>, nil]
26
+ #
27
+ def dequeue_recv
28
+ @recv_queue.dequeue
29
+ end
30
+
31
+
32
+ # Wakes a blocked {#dequeue_recv} with a nil sentinel. Called by
33
+ # Engine on close or fatal-error propagation.
34
+ #
35
+ # @return [void]
36
+ #
37
+ def unblock_recv
38
+ @recv_queue.enqueue(nil)
39
+ end
40
+
41
+
25
42
  # @param connection [Connection]
26
43
  #
27
44
  def connection_added(connection)
28
- add_fair_recv_connection(connection)
45
+ task = @engine.start_recv_pump(connection, @recv_queue)
46
+ @tasks << task if task
29
47
  end
30
48
 
31
49
 
32
50
  # @param connection [Connection]
33
51
  #
34
52
  def connection_removed(connection)
35
- @recv_queue.remove_queue(connection)
36
- # recv pump stops on EOFError
53
+ # recv pump stops on EOFError via its connection barrier
37
54
  end
38
55
 
39
56
 
@@ -9,12 +9,10 @@ module OMQ
9
9
  # on send.
10
10
  #
11
11
  class Rep
12
- include FairRecv
13
-
14
12
  EMPTY_FRAME = "".b.freeze
15
13
 
16
14
 
17
- # @return [FairQueue]
15
+ # @return [Async::LimitedQueue]
18
16
  #
19
17
  attr_reader :recv_queue
20
18
 
@@ -23,7 +21,7 @@ module OMQ
23
21
  #
24
22
  def initialize(engine)
25
23
  @engine = engine
26
- @recv_queue = FairQueue.new
24
+ @recv_queue = Routing.build_queue(engine.options.recv_hwm, :block)
27
25
  @pending_replies = []
28
26
  @conn_queues = {} # connection => per-connection send queue
29
27
  @conn_send_tasks = {} # connection => send pump task
@@ -31,10 +29,28 @@ module OMQ
31
29
  end
32
30
 
33
31
 
32
+ # Dequeues the next received message. Blocks until one is available.
33
+ #
34
+ # @return [Array<String>, nil]
35
+ #
36
+ def dequeue_recv
37
+ @recv_queue.dequeue
38
+ end
39
+
40
+
41
+ # Wakes a blocked {#dequeue_recv} with a nil sentinel.
42
+ #
43
+ # @return [void]
44
+ #
45
+ def unblock_recv
46
+ @recv_queue.enqueue(nil)
47
+ end
48
+
49
+
34
50
  # @param connection [Connection]
35
51
  #
36
52
  def connection_added(connection)
37
- add_fair_recv_connection(connection) do |msg|
53
+ task = @engine.start_recv_pump(connection, @recv_queue) do |msg|
38
54
  delimiter = msg.index { |p| p.empty? } || msg.size
39
55
  envelope = msg[0, delimiter]
40
56
  body = msg[(delimiter + 1)..] || []
@@ -42,6 +58,7 @@ module OMQ
42
58
  @pending_replies << { conn: connection, envelope: envelope }
43
59
  body
44
60
  end
61
+ @tasks << task if task
45
62
 
46
63
  q = Routing.build_queue(@engine.options.send_hwm, :block)
47
64
  @conn_queues[connection] = q
@@ -53,7 +70,6 @@ module OMQ
53
70
  #
54
71
  def connection_removed(connection)
55
72
  @pending_replies.reject! { |r| r[:conn] == connection }
56
- @recv_queue.remove_queue(connection)
57
73
  @conn_queues.delete(connection)
58
74
  @conn_send_tasks.delete(connection)&.stop
59
75
  end
@@ -8,13 +8,12 @@ module OMQ
8
8
  #
9
9
  class Req
10
10
  include RoundRobin
11
- include FairRecv
12
11
 
13
12
  # Shared frozen empty binary string to avoid repeated allocations.
14
13
  EMPTY_BINARY = ::Protocol::ZMTP::Codec::EMPTY_BINARY
15
14
 
16
15
 
17
- # @return [FairQueue]
16
+ # @return [Async::LimitedQueue]
18
17
  #
19
18
  attr_reader :recv_queue
20
19
 
@@ -23,20 +22,40 @@ module OMQ
23
22
  #
24
23
  def initialize(engine)
25
24
  @engine = engine
26
- @recv_queue = FairQueue.new
25
+ @recv_queue = Routing.build_queue(engine.options.recv_hwm, :block)
27
26
  @tasks = []
28
27
  @state = :ready # :ready or :waiting_reply
29
28
  init_round_robin(engine)
30
29
  end
31
30
 
32
31
 
32
+ # Dequeues the next received message. Blocks until one is available.
33
+ #
34
+ # @return [Array<String>, nil]
35
+ #
36
+ def dequeue_recv
37
+ @recv_queue.dequeue
38
+ end
39
+
40
+
41
+ # Wakes a blocked {#dequeue_recv} with a nil sentinel.
42
+ #
43
+ # @return [void]
44
+ #
45
+ def unblock_recv
46
+ @recv_queue.enqueue(nil)
47
+ end
48
+
49
+
33
50
  # @param connection [Connection]
34
51
  #
35
52
  def connection_added(connection)
36
- add_fair_recv_connection(connection) do |msg|
53
+ task = @engine.start_recv_pump(connection, @recv_queue) do |msg|
37
54
  @state = :ready
38
55
  msg.first&.empty? ? msg[1..] : msg
39
56
  end
57
+
58
+ @tasks << task if task
40
59
  add_round_robin_send_connection(connection)
41
60
  end
42
61
 
@@ -45,7 +64,6 @@ module OMQ
45
64
  #
46
65
  def connection_removed(connection)
47
66
  @connections.delete(connection)
48
- @recv_queue.remove_queue(connection)
49
67
  remove_round_robin_send_connection(connection)
50
68
  end
51
69
 
@@ -159,6 +159,7 @@ module OMQ
159
159
 
160
160
  def drain_send_queue_capped(batch)
161
161
  bytes = batch_bytes(batch.first)
162
+
162
163
  while batch.size < BATCH_MSG_CAP && bytes < BATCH_BYTE_CAP
163
164
  msg = @send_queue.dequeue(timeout: 0)
164
165
  break unless msg
@@ -11,10 +11,7 @@ module OMQ
11
11
  # routing identity on send.
12
12
  #
13
13
  class Router
14
- include FairRecv
15
-
16
-
17
- # @return [FairQueue]
14
+ # @return [Async::LimitedQueue]
18
15
  #
19
16
  attr_reader :recv_queue
20
17
 
@@ -23,7 +20,7 @@ module OMQ
23
20
  #
24
21
  def initialize(engine)
25
22
  @engine = engine
26
- @recv_queue = FairQueue.new
23
+ @recv_queue = Routing.build_queue(engine.options.recv_hwm, :block)
27
24
  @connections_by_identity = {}
28
25
  @identity_by_connection = {}
29
26
  @conn_queues = {} # connection => per-connection send queue
@@ -32,6 +29,24 @@ module OMQ
32
29
  end
33
30
 
34
31
 
32
+ # Dequeues the next received message. Blocks until one is available.
33
+ #
34
+ # @return [Array<String>, nil]
35
+ #
36
+ def dequeue_recv
37
+ @recv_queue.dequeue
38
+ end
39
+
40
+
41
+ # Wakes a blocked {#dequeue_recv} with a nil sentinel.
42
+ #
43
+ # @return [void]
44
+ #
45
+ def unblock_recv
46
+ @recv_queue.enqueue(nil)
47
+ end
48
+
49
+
35
50
  # @param connection [Connection]
36
51
  #
37
52
  def connection_added(connection)
@@ -40,7 +55,8 @@ module OMQ
40
55
  @connections_by_identity[identity] = connection
41
56
  @identity_by_connection[connection] = identity
42
57
 
43
- add_fair_recv_connection(connection) { |msg| [identity, *msg] }
58
+ task = @engine.start_recv_pump(connection, @recv_queue) { |msg| [identity, *msg] }
59
+ @tasks << task if task
44
60
 
45
61
  q = Routing.build_queue(@engine.options.send_hwm, :block)
46
62
  @conn_queues[connection] = q
@@ -53,7 +69,6 @@ module OMQ
53
69
  def connection_removed(connection)
54
70
  identity = @identity_by_connection.delete(connection)
55
71
  @connections_by_identity.delete(identity) if identity
56
- @recv_queue.remove_queue(connection)
57
72
  @conn_queues.delete(connection)
58
73
  @conn_send_tasks.delete(connection)&.stop
59
74
  end
@@ -8,7 +8,7 @@ module OMQ
8
8
  #
9
9
  class Sub
10
10
 
11
- # @return [FairQueue]
11
+ # @return [Async::LimitedQueue]
12
12
  #
13
13
  attr_reader :recv_queue
14
14
 
@@ -18,21 +18,27 @@ module OMQ
18
18
  def initialize(engine)
19
19
  @engine = engine
20
20
  @connections = Set.new
21
- @recv_queue = FairQueue.new
21
+ @recv_queue = Routing.build_queue(engine.options.recv_hwm, :block)
22
22
  @subscriptions = Set.new
23
23
  @tasks = []
24
24
  end
25
25
 
26
26
 
27
- # Engine-facing recv contract. Delegates to the FairQueue.
27
+ # Dequeues the next received message. Blocks until one is available.
28
+ #
29
+ # @return [Array<String>, nil]
28
30
  #
29
31
  def dequeue_recv
30
32
  @recv_queue.dequeue
31
33
  end
32
34
 
33
35
 
36
+ # Wakes a blocked {#dequeue_recv} with a nil sentinel.
37
+ #
38
+ # @return [void]
39
+ #
34
40
  def unblock_recv
35
- @recv_queue.push(nil)
41
+ @recv_queue.enqueue(nil)
36
42
  end
37
43
 
38
44
 
@@ -40,13 +46,12 @@ module OMQ
40
46
  #
41
47
  def connection_added(connection)
42
48
  @connections << connection
49
+
43
50
  @subscriptions.each do |prefix|
44
51
  connection.send_command(Protocol::ZMTP::Codec::Command.subscribe(prefix))
45
52
  end
46
- conn_q = Routing.build_queue(@engine.options.recv_hwm, @engine.options.on_mute)
47
- signaling = SignalingQueue.new(conn_q, @recv_queue)
48
- @recv_queue.add_queue(connection, conn_q)
49
- task = @engine.start_recv_pump(connection, signaling)
53
+
54
+ task = @engine.start_recv_pump(connection, @recv_queue)
50
55
  @tasks << task if task
51
56
  end
52
57
 
@@ -55,7 +60,6 @@ module OMQ
55
60
  #
56
61
  def connection_removed(connection)
57
62
  @connections.delete(connection)
58
- @recv_queue.remove_queue(connection)
59
63
  end
60
64
 
61
65
 
@@ -8,8 +8,8 @@ module OMQ
8
8
  # the application as data frames: \x01 + prefix for subscribe,
9
9
  # \x00 + prefix for unsubscribe.
10
10
  #
11
- # The recv_queue is a simple bounded queue (not a FairQueue) because
12
- # messages come from subscription commands, not from peer data pumps.
11
+ # The recv_queue is a simple bounded queue because messages come from
12
+ # subscription commands, not from peer data pumps.
13
13
  #
14
14
  class XPub
15
15
  include FanOut
@@ -25,6 +25,7 @@ module OMQ
25
25
  @engine = engine
26
26
  @recv_queue = Routing.build_queue(engine.options.recv_hwm, :block)
27
27
  @tasks = []
28
+
28
29
  init_fan_out(engine)
29
30
  end
30
31
 
@@ -10,7 +10,7 @@ module OMQ
10
10
  #
11
11
  class XSub
12
12
 
13
- # @return [FairQueue]
13
+ # @return [Async::LimitedQueue]
14
14
  #
15
15
  attr_reader :recv_queue
16
16
 
@@ -20,22 +20,28 @@ module OMQ
20
20
  def initialize(engine)
21
21
  @engine = engine
22
22
  @connections = Set.new
23
- @recv_queue = FairQueue.new
23
+ @recv_queue = Routing.build_queue(engine.options.recv_hwm, :block)
24
24
  @conn_queues = {} # connection => per-connection send queue
25
25
  @conn_send_tasks = {} # connection => send pump task
26
26
  @tasks = []
27
27
  end
28
28
 
29
29
 
30
- # Engine-facing recv contract. Delegates to the FairQueue.
30
+ # Dequeues the next received message. Blocks until one is available.
31
+ #
32
+ # @return [Array<String>, nil]
31
33
  #
32
34
  def dequeue_recv
33
35
  @recv_queue.dequeue
34
36
  end
35
37
 
36
38
 
39
+ # Wakes a blocked {#dequeue_recv} with a nil sentinel.
40
+ #
41
+ # @return [void]
42
+ #
37
43
  def unblock_recv
38
- @recv_queue.push(nil)
44
+ @recv_queue.enqueue(nil)
39
45
  end
40
46
 
41
47
 
@@ -44,10 +50,7 @@ module OMQ
44
50
  def connection_added(connection)
45
51
  @connections << connection
46
52
 
47
- conn_q = Routing.build_queue(@engine.options.recv_hwm, @engine.options.on_mute)
48
- signaling = SignalingQueue.new(conn_q, @recv_queue)
49
- @recv_queue.add_queue(connection, conn_q)
50
- task = @engine.start_recv_pump(connection, signaling)
53
+ task = @engine.start_recv_pump(connection, @recv_queue)
51
54
  @tasks << task if task
52
55
 
53
56
  q = Routing.build_queue(@engine.options.send_hwm, :block)
@@ -60,7 +63,6 @@ module OMQ
60
63
  #
61
64
  def connection_removed(connection)
62
65
  @connections.delete(connection)
63
- @recv_queue.remove_queue(connection)
64
66
  @conn_queues.delete(connection)
65
67
  @conn_send_tasks.delete(connection)&.stop
66
68
  end
data/lib/omq/routing.rb CHANGED
@@ -4,8 +4,6 @@ require "async"
4
4
  require "async/queue"
5
5
  require "async/limited_queue"
6
6
  require_relative "drop_queue"
7
- require_relative "routing/fair_queue"
8
- require_relative "routing/fair_recv"
9
7
  require_relative "routing/conn_send_pump"
10
8
 
11
9
  module OMQ
data/lib/omq/socket.rb CHANGED
@@ -298,6 +298,11 @@ module OMQ
298
298
  def attach_endpoints(endpoints, default:)
299
299
  return unless endpoints
300
300
 
301
+ if endpoints.is_a?(Array)
302
+ endpoints.each { |ep| attach_endpoints(ep, default: default) }
303
+ return
304
+ end
305
+
301
306
  case endpoints
302
307
  when /\A@(.+)\z/
303
308
  bind($1)
data/lib/omq/version.rb CHANGED
@@ -1,5 +1,5 @@
1
1
  # frozen_string_literal: true
2
2
 
3
3
  module OMQ
4
- VERSION = "0.20.0"
4
+ VERSION = "0.21.0"
5
5
  end
metadata CHANGED
@@ -1,7 +1,7 @@
1
1
  --- !ruby/object:Gem::Specification
2
2
  name: omq
3
3
  version: !ruby/object:Gem::Version
4
- version: 0.20.0
4
+ version: 0.21.0
5
5
  platform: ruby
6
6
  authors:
7
7
  - Patrik Wenger
@@ -86,8 +86,6 @@ files:
86
86
  - lib/omq/routing.rb
87
87
  - lib/omq/routing/conn_send_pump.rb
88
88
  - lib/omq/routing/dealer.rb
89
- - lib/omq/routing/fair_queue.rb
90
- - lib/omq/routing/fair_recv.rb
91
89
  - lib/omq/routing/fan_out.rb
92
90
  - lib/omq/routing/pair.rb
93
91
  - lib/omq/routing/pub.rb
@@ -1,197 +0,0 @@
1
- # frozen_string_literal: true
2
-
3
- module OMQ
4
- module Routing
5
- # Per-connection recv queue aggregator.
6
- #
7
- # Maintains one bounded queue per connected peer. #dequeue
8
- # returns the next available message from any peer in fair
9
- # round-robin order, blocking until one arrives.
10
- #
11
- # Recv pumps do not enqueue directly — they write through a
12
- # SignalingQueue wrapper, which also wakes a blocked #dequeue.
13
- #
14
- class FairQueue
15
- # Creates an empty fair queue with no per-connection queues.
16
- #
17
- def initialize
18
- @queues = [] # ordered list of per-connection inner queues
19
- @drain = [] # orphaned queues, drained before active queues
20
- @mapping = {} # connection => inner queue
21
- @cycle = @queues.cycle # live reference — sees adds/removes
22
- @condition = Async::Condition.new
23
- @pending = 0 # signals received before #dequeue waits
24
- @closed = false
25
- end
26
-
27
-
28
- # Registers a per-connection queue. Called when a connection is added.
29
- #
30
- # @param conn [Connection]
31
- # @param q [Async::LimitedQueue]
32
- #
33
- def add_queue(conn, q)
34
- @mapping[conn] = q
35
- @queues << q
36
- end
37
-
38
-
39
- # Removes the per-connection queue for a disconnected peer.
40
- #
41
- # If the queue still has pending messages it moves to the
42
- # priority drain list so those messages are consumed before
43
- # any active connection's messages — preserving FIFO for
44
- # sequential connections.
45
- #
46
- # @param conn [Connection]
47
- #
48
- def remove_queue(conn)
49
- q = @mapping.delete(conn)
50
- return unless q
51
- @queues.delete(q)
52
- @drain << q unless q.empty?
53
- end
54
-
55
-
56
- # Wakes a blocked #dequeue. Called by SignalingQueue after each enqueue.
57
- #
58
- def signal
59
- @pending += 1
60
- @condition.signal
61
- end
62
-
63
-
64
- # Returns the next message from any per-connection queue, in fair
65
- # round-robin order. Blocks until a message is available.
66
- #
67
- # Pass +timeout: 0+ for a non-blocking poll (returns nil immediately
68
- # if no messages are available).
69
- #
70
- # @param timeout [Numeric, nil] 0 = non-blocking, nil = block forever
71
- # @return [Array<String>, nil]
72
- #
73
- def dequeue(timeout: nil)
74
- return try_dequeue if timeout == 0
75
-
76
- loop do
77
- if @closed && @drain.empty? && @queues.all? { |q| q.empty? }
78
- return nil
79
- end
80
-
81
- msg = try_dequeue
82
- return msg if msg
83
-
84
- if @pending > 0
85
- @pending -= 1
86
- next
87
- end
88
-
89
- @condition.wait
90
- end
91
- end
92
-
93
-
94
- # Injects a nil sentinel to unblock a waiting #dequeue.
95
- # Called by Engine on close or fatal error.
96
- #
97
- def push(nil_sentinel)
98
- @closed = true
99
- @condition.signal
100
- end
101
-
102
-
103
- # @return [Boolean]
104
- #
105
- def empty?
106
- @drain.empty? && @queues.all? { |q| q.empty? }
107
- end
108
-
109
-
110
- private
111
-
112
-
113
- # Drains orphaned queues first (preserves FIFO for disconnected
114
- # peers), then tries each active queue once in round-robin order.
115
- #
116
- def try_dequeue
117
- # Priority: drain orphaned queues before serving active ones
118
- until @drain.empty?
119
- msg = @drain.first.dequeue(timeout: 0)
120
- if msg
121
- return msg
122
- else
123
- @drain.shift
124
- end
125
- end
126
-
127
- # Single-connection fast path: skip Enumerator#next entirely.
128
- # The vast majority of sockets have exactly one peer.
129
- if @queues.size == 1
130
- return @queues.first.dequeue(timeout: 0)
131
- end
132
-
133
- @queues.size.times do
134
- q = begin
135
- @cycle.next
136
- rescue StopIteration
137
- @cycle = @queues.cycle
138
- break
139
- end
140
- msg = q.dequeue(timeout: 0)
141
- return msg if msg
142
- end
143
-
144
- nil
145
- end
146
- end
147
-
148
-
149
- # Wraps a per-connection bounded queue so that each #enqueue also
150
- # signals the FairQueue to wake a blocked #dequeue.
151
- #
152
- class SignalingQueue
153
- # @param inner [Async::LimitedQueue] the per-connection bounded queue
154
- # @param fair_queue [FairQueue] the parent fair queue to signal on enqueue
155
- #
156
- def initialize(inner, fair_queue)
157
- @inner = inner
158
- @fair = fair_queue
159
- end
160
-
161
-
162
- # Enqueues a message and signals the fair queue.
163
- #
164
- # @param msg [Array<String>]
165
- # @return [void]
166
- #
167
- def enqueue(msg)
168
- @inner.enqueue(msg)
169
- @fair.signal
170
- end
171
-
172
-
173
- # @param timeout [Numeric, nil] dequeue timeout
174
- # @return [Array<String>, nil]
175
- #
176
- def dequeue(timeout: nil)
177
- @inner.dequeue(timeout: timeout)
178
- end
179
-
180
-
181
- # @return [Boolean]
182
- #
183
- def empty?
184
- @inner.empty?
185
- end
186
-
187
-
188
- # @param item [Object, nil]
189
- # @return [void]
190
- #
191
- def push(item)
192
- @inner.push(item)
193
- end
194
-
195
- end
196
- end
197
- end
@@ -1,53 +0,0 @@
1
- # frozen_string_literal: true
2
-
3
- module OMQ
4
- module Routing
5
- # Mixin that adds per-connection recv queue setup for fair-queued sockets.
6
- #
7
- # Including classes must have @engine, @recv_queue (FairQueue), and @tasks.
8
- #
9
- module FairRecv
10
- # Dequeues the next received message. Blocks until one is available.
11
- # Engine-facing contract — Engine must not touch @recv_queue directly.
12
- #
13
- # @return [Array<String>, nil]
14
- #
15
- def dequeue_recv
16
- @recv_queue.dequeue
17
- end
18
-
19
-
20
- # Wakes a blocked {#dequeue_recv} with a nil sentinel. Called by
21
- # Engine on close (close_read) or fatal-error propagation.
22
- #
23
- # @return [void]
24
- #
25
- def unblock_recv
26
- @recv_queue.push(nil)
27
- end
28
-
29
-
30
- private
31
-
32
- # Creates a per-connection recv queue, registers it with @recv_queue,
33
- # and starts a recv pump for the connection. Called from #connection_added.
34
- #
35
- # @param conn [Connection]
36
- # @yield [msg] optional per-message transform
37
- #
38
- def add_fair_recv_connection(conn, &transform)
39
- conn_q = Routing.build_queue(@engine.options.recv_hwm, :block)
40
- signaling = SignalingQueue.new(conn_q, @recv_queue)
41
-
42
- @recv_queue.add_queue(conn, conn_q)
43
-
44
- task = @engine.start_recv_pump(conn, signaling, &transform)
45
-
46
- if task
47
- @tasks << task
48
- end
49
- end
50
-
51
- end
52
- end
53
- end