omq 0.14.1 → 0.15.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/CHANGELOG.md +54 -0
- data/lib/omq/engine/connection_setup.rb +1 -1
- data/lib/omq/engine/reconnect.rb +9 -5
- data/lib/omq/engine/recv_pump.rb +4 -2
- data/lib/omq/engine.rb +19 -1
- data/lib/omq/options.rb +8 -1
- data/lib/omq/routing/conn_send_pump.rb +1 -0
- data/lib/omq/routing/dealer.rb +0 -1
- data/lib/omq/routing/fair_queue.rb +21 -16
- data/lib/omq/routing/fan_out.rb +5 -1
- data/lib/omq/routing/pair.rb +10 -3
- data/lib/omq/routing/push.rb +0 -1
- data/lib/omq/routing/req.rb +0 -1
- data/lib/omq/routing/round_robin.rb +34 -9
- data/lib/omq/routing/staging_queue.rb +60 -0
- data/lib/omq/routing.rb +1 -0
- data/lib/omq/single_frame.rb +23 -0
- data/lib/omq/socket.rb +4 -1
- data/lib/omq/transport/ipc.rb +13 -2
- data/lib/omq/transport/tcp.rb +14 -2
- data/lib/omq/version.rb +1 -1
- data/lib/omq.rb +6 -0
- metadata +3 -1
checksums.yaml
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
SHA256:
|
|
3
|
-
metadata.gz:
|
|
4
|
-
data.tar.gz:
|
|
3
|
+
metadata.gz: 20e7575603dbe5f8f68e141aa0038a9e2cf7c14e31d9da0d760d7a2b13c99ab6
|
|
4
|
+
data.tar.gz: 0a0b985e79f73a47a30576ed076aca0b4a2eb59b8caee7b264a2afa54c314c54
|
|
5
5
|
SHA512:
|
|
6
|
-
metadata.gz:
|
|
7
|
-
data.tar.gz:
|
|
6
|
+
metadata.gz: 6580a6dbc766bfc737616bbcf427c05b0505fddcc167d37739b6c7750505749a955bd6df5c070126f4e8cfde60abe03a23549e5f179d4d4aa1dee370d7de5716
|
|
7
|
+
data.tar.gz: 73a81ef7e80ea2589dfb11502927369ca0dbffbeecfc867a4c5ce6c3d850ef413028be832e30615ae3b7f3983c611c8dfd3693e076b78f527f8abcff624b520a
|
data/CHANGELOG.md
CHANGED
|
@@ -1,7 +1,61 @@
|
|
|
1
1
|
# Changelog
|
|
2
2
|
|
|
3
|
+
## 0.15.0 — 2026-04-07
|
|
4
|
+
|
|
5
|
+
- **Fix pipe FIFO ordering** — messages from sequential source batches could
|
|
6
|
+
interleave when a connection dropped and reconnected. `FairQueue` now moves
|
|
7
|
+
orphaned per-connection queues to a priority drain list, ensuring all buffered
|
|
8
|
+
messages from a disconnected peer are consumed before any new peer's messages.
|
|
9
|
+
- **Fix lost messages on disconnect** — `RoundRobin#remove_round_robin_send_connection`
|
|
10
|
+
now drains the per-connection send queue back to staging before closing it, and
|
|
11
|
+
the send pump re-stages its in-flight batch on `CONNECTION_LOST`. Previously
|
|
12
|
+
messages in the per-connection queue or mid-batch were silently dropped.
|
|
13
|
+
- **Fix `next_connection` deadlock** — when the round-robin cycle exhausted with
|
|
14
|
+
connections still present, a new unresolved `Async::Promise` was created
|
|
15
|
+
unconditionally, blocking the sender forever. Now only creates a new promise
|
|
16
|
+
when `@connections` is actually empty.
|
|
17
|
+
- **Fix staging drain race** — `add_round_robin_send_connection` now appends to
|
|
18
|
+
`@connections` after draining staging (not before), preventing the pipe loop
|
|
19
|
+
from bypassing staging during drain. A second drain pass catches any message
|
|
20
|
+
that squeezed in during the first.
|
|
21
|
+
- **Fix `handshake_succeeded` event ordering** — the monitor event is now emitted
|
|
22
|
+
before `connection_added` (which may yield during drain), so it always appears
|
|
23
|
+
before any `message_sent` events on that connection.
|
|
24
|
+
- **Fix send pump `Async::Stop` preventing reconnect** — `remove_round_robin_send_connection`
|
|
25
|
+
no longer calls `task.stop` on the send pump. Instead it closes the queue and
|
|
26
|
+
lets the pump detect nil, avoiding `Async::Stop` propagation that prevented
|
|
27
|
+
`maybe_reconnect` from running.
|
|
28
|
+
- **Add `StagingQueue`** — bounded FIFO queue with `#prepend` for re-staging
|
|
29
|
+
failed messages at the front. Replaces raw `Async::LimitedQueue` in
|
|
30
|
+
`RoundRobin` and `Pair` routing strategies.
|
|
31
|
+
- **Add `SingleFrame` mixin to core** — moved from 5 duplicate copies across
|
|
32
|
+
RFC gems to `OMQ::SingleFrame`, eliminating method redefinition warnings.
|
|
33
|
+
- **Add `SO_SNDBUF` / `SO_RCVBUF` socket options** — `Options#sndbuf` and
|
|
34
|
+
`Options#rcvbuf` set kernel buffer sizes on TCP and IPC sockets (both
|
|
35
|
+
accepted and connected).
|
|
36
|
+
- **Add verbose monitor events** — `Socket#monitor(verbose: true)` emits
|
|
37
|
+
`:message_sent` and `:message_received` events via `Engine#emit_verbose_monitor_event`.
|
|
38
|
+
Allocation-free when verbose is off.
|
|
39
|
+
- **Add `OMQ::DEBUG` flag** — when `OMQ_DEBUG` is set, transport accept loops
|
|
40
|
+
print unexpected exceptions to stderr.
|
|
41
|
+
- **Fix `Pair` re-staging on disconnect** — `Pair#connection_removed` now drains
|
|
42
|
+
the per-connection send queue back to staging, and the send pump re-stages its
|
|
43
|
+
batch on `CONNECTION_LOST`.
|
|
44
|
+
|
|
3
45
|
## 0.14.1 — 2026-04-07
|
|
4
46
|
|
|
47
|
+
- **Fix PUSH send queue deadlock on disconnect** — when a peer disconnected
|
|
48
|
+
while a fiber was blocked on a full per-connection send queue (low `send_hwm`),
|
|
49
|
+
the fiber hung forever. Now closes the queue on disconnect, raising
|
|
50
|
+
`ClosedError` which re-routes the message to staging. Also reorders
|
|
51
|
+
`add_round_robin_send_connection` to start the send pump before draining
|
|
52
|
+
staging, preventing deadlock with small queues.
|
|
53
|
+
- **Fix reconnect backoff for plain Numeric** — `#next_delay` incorrectly
|
|
54
|
+
doubled the delay even when `reconnect_interval` was a plain Numeric. Now
|
|
55
|
+
only Range triggers exponential backoff; a fixed Numeric returns the same
|
|
56
|
+
interval every retry.
|
|
57
|
+
- **Default `reconnect_interval` changed to `0.1..1.0`** — uses exponential
|
|
58
|
+
backoff (100 ms → 1 s cap) by default instead of a fixed 100 ms.
|
|
5
59
|
- **Fix per-connection task tree** — recv pump, heartbeat, and reaper tasks
|
|
6
60
|
were spawned under `@parent_task` (socket-level) instead of the connection
|
|
7
61
|
task. When `@parent_task` finished before a late connection completed its
|
|
@@ -37,8 +37,8 @@ module OMQ
|
|
|
37
37
|
conn.handshake!
|
|
38
38
|
Heartbeat.start(Async::Task.current, conn, @engine.options, @engine.tasks)
|
|
39
39
|
conn = @engine.connection_wrapper.call(conn) if @engine.connection_wrapper
|
|
40
|
-
register(conn, endpoint, done)
|
|
41
40
|
@engine.emit_monitor_event(:handshake_succeeded, endpoint: endpoint)
|
|
41
|
+
register(conn, endpoint, done)
|
|
42
42
|
conn
|
|
43
43
|
rescue Protocol::ZMTP::Error, *CONNECTION_LOST => error
|
|
44
44
|
@engine.emit_monitor_event(:handshake_failed, endpoint: endpoint, detail: { error: error })
|
data/lib/omq/engine/reconnect.rb
CHANGED
|
@@ -71,11 +71,15 @@ module OMQ
|
|
|
71
71
|
|
|
72
72
|
|
|
73
73
|
def next_delay(delay, max_delay)
|
|
74
|
-
ri
|
|
75
|
-
|
|
76
|
-
|
|
77
|
-
|
|
78
|
-
|
|
74
|
+
ri = @options.reconnect_interval
|
|
75
|
+
if ri.is_a?(Range)
|
|
76
|
+
delay = delay * 2
|
|
77
|
+
delay = [delay, max_delay].min if max_delay
|
|
78
|
+
delay = ri.begin if delay == 0
|
|
79
|
+
delay
|
|
80
|
+
else
|
|
81
|
+
ri
|
|
82
|
+
end
|
|
79
83
|
end
|
|
80
84
|
end
|
|
81
85
|
end
|
data/lib/omq/engine/recv_pump.rb
CHANGED
|
@@ -68,7 +68,7 @@ module OMQ
|
|
|
68
68
|
|
|
69
69
|
|
|
70
70
|
def start_with_transform(parent_task, transform)
|
|
71
|
-
conn, recv_queue, count_bytes = @conn, @recv_queue, @count_bytes
|
|
71
|
+
conn, recv_queue, engine, count_bytes = @conn, @recv_queue, @engine, @count_bytes
|
|
72
72
|
|
|
73
73
|
parent_task.async(transient: true, annotation: "recv pump") do |task|
|
|
74
74
|
loop do
|
|
@@ -78,6 +78,7 @@ module OMQ
|
|
|
78
78
|
msg = conn.receive_message
|
|
79
79
|
msg = transform.call(msg).freeze
|
|
80
80
|
recv_queue.enqueue(msg)
|
|
81
|
+
engine.emit_verbose_monitor_event(:message_received, parts: msg)
|
|
81
82
|
count += 1
|
|
82
83
|
bytes += msg.sum(&:bytesize) if count_bytes
|
|
83
84
|
end
|
|
@@ -93,7 +94,7 @@ module OMQ
|
|
|
93
94
|
|
|
94
95
|
|
|
95
96
|
def start_direct(parent_task)
|
|
96
|
-
conn, recv_queue, count_bytes = @conn, @recv_queue, @count_bytes
|
|
97
|
+
conn, recv_queue, engine, count_bytes = @conn, @recv_queue, @engine, @count_bytes
|
|
97
98
|
|
|
98
99
|
parent_task.async(transient: true, annotation: "recv pump") do |task|
|
|
99
100
|
loop do
|
|
@@ -102,6 +103,7 @@ module OMQ
|
|
|
102
103
|
while count < FAIRNESS_MESSAGES && bytes < FAIRNESS_BYTES
|
|
103
104
|
msg = conn.receive_message
|
|
104
105
|
recv_queue.enqueue(msg)
|
|
106
|
+
engine.emit_verbose_monitor_event(:message_received, parts: msg)
|
|
105
107
|
count += 1
|
|
106
108
|
bytes += msg.sum(&:bytesize) if count_bytes
|
|
107
109
|
end
|
data/lib/omq/engine.rb
CHANGED
|
@@ -78,6 +78,7 @@ module OMQ
|
|
|
78
78
|
@on_io_thread = false
|
|
79
79
|
@fatal_error = nil
|
|
80
80
|
@monitor_queue = nil
|
|
81
|
+
@verbose_monitor = false
|
|
81
82
|
end
|
|
82
83
|
|
|
83
84
|
|
|
@@ -95,6 +96,7 @@ module OMQ
|
|
|
95
96
|
# @param value [Async::Queue, nil] queue for monitor events
|
|
96
97
|
#
|
|
97
98
|
attr_writer :reconnect_enabled, :monitor_queue
|
|
99
|
+
attr_accessor :verbose_monitor
|
|
98
100
|
|
|
99
101
|
# @return [Boolean] true if the engine has been closed
|
|
100
102
|
#
|
|
@@ -222,9 +224,9 @@ module OMQ
|
|
|
222
224
|
def connection_ready(pipe, endpoint: nil)
|
|
223
225
|
pipe = @connection_wrapper.call(pipe) if @connection_wrapper
|
|
224
226
|
@connections[pipe] = ConnectionRecord.new(endpoint: endpoint, done: nil)
|
|
227
|
+
emit_monitor_event(:handshake_succeeded, endpoint: endpoint)
|
|
225
228
|
@routing.connection_added(pipe)
|
|
226
229
|
@peer_connected.resolve(pipe)
|
|
227
|
-
emit_monitor_event(:handshake_succeeded, endpoint: endpoint)
|
|
228
230
|
end
|
|
229
231
|
|
|
230
232
|
|
|
@@ -404,6 +406,20 @@ module OMQ
|
|
|
404
406
|
end
|
|
405
407
|
|
|
406
408
|
|
|
409
|
+
# Emits a verbose-only monitor event (e.g. message traces).
|
|
410
|
+
# Only emitted when {Socket#monitor} was called with +verbose: true+.
|
|
411
|
+
# Uses +**detail+ to avoid Hash allocation when verbose is off.
|
|
412
|
+
#
|
|
413
|
+
# @param type [Symbol] event type (e.g. :message_sent, :message_received)
|
|
414
|
+
# @param detail [Hash] extra context forwarded as keyword args
|
|
415
|
+
# @return [void]
|
|
416
|
+
#
|
|
417
|
+
def emit_verbose_monitor_event(type, **detail)
|
|
418
|
+
return unless @verbose_monitor
|
|
419
|
+
emit_monitor_event(type, detail: detail)
|
|
420
|
+
end
|
|
421
|
+
|
|
422
|
+
|
|
407
423
|
# Looks up the transport module for an endpoint URI.
|
|
408
424
|
#
|
|
409
425
|
# @param endpoint [String] endpoint URI (e.g. "tcp://...", "inproc://...")
|
|
@@ -423,6 +439,8 @@ module OMQ
|
|
|
423
439
|
done = Async::Promise.new
|
|
424
440
|
conn = ConnectionSetup.run(io, self, as_server: as_server, endpoint: endpoint, done: done)
|
|
425
441
|
done.wait
|
|
442
|
+
rescue Async::Queue::ClosedError
|
|
443
|
+
# connection dropped during drain — message re-staged
|
|
426
444
|
rescue Protocol::ZMTP::Error, *CONNECTION_LOST
|
|
427
445
|
# handshake failed or connection lost — subtree cleaned up
|
|
428
446
|
ensure
|
data/lib/omq/options.rb
CHANGED
|
@@ -20,12 +20,14 @@ module OMQ
|
|
|
20
20
|
@router_mandatory = false
|
|
21
21
|
@read_timeout = nil # seconds, nil = no timeout
|
|
22
22
|
@write_timeout = nil
|
|
23
|
-
@reconnect_interval = 0.1
|
|
23
|
+
@reconnect_interval = 0.1..1.0 # seconds; Range sets backoff min..max
|
|
24
24
|
@heartbeat_interval = nil # seconds, nil = disabled
|
|
25
25
|
@heartbeat_ttl = nil # seconds, nil = use heartbeat_interval
|
|
26
26
|
@heartbeat_timeout = nil # seconds, nil = use heartbeat_interval
|
|
27
27
|
@max_message_size = 1 << 20 # bytes (1 MiB default)
|
|
28
28
|
@conflate = false
|
|
29
|
+
@sndbuf = nil # bytes, nil = OS default
|
|
30
|
+
@rcvbuf = nil # bytes, nil = OS default
|
|
29
31
|
@on_mute = :block # :block, :drop_newest, :drop_oldest
|
|
30
32
|
@mechanism = Protocol::ZMTP::Mechanism::Null.new
|
|
31
33
|
@qos = 0 # 0 = fire-and-forget, 1 = at-least-once (see omq-qos gem)
|
|
@@ -58,6 +60,10 @@ module OMQ
|
|
|
58
60
|
# @return [Numeric, nil] time without traffic before closing (nil = use heartbeat_interval)
|
|
59
61
|
# @!attribute max_message_size
|
|
60
62
|
# @return [Integer, nil] maximum message size in bytes
|
|
63
|
+
# @!attribute sndbuf
|
|
64
|
+
# @return [Integer, nil] SO_SNDBUF size in bytes (nil = OS default)
|
|
65
|
+
# @!attribute rcvbuf
|
|
66
|
+
# @return [Integer, nil] SO_RCVBUF size in bytes (nil = OS default)
|
|
61
67
|
# @!attribute on_mute
|
|
62
68
|
# @return [Symbol] mute strategy (:block, :drop_newest, :drop_oldest)
|
|
63
69
|
# @!attribute mechanism
|
|
@@ -72,6 +78,7 @@ module OMQ
|
|
|
72
78
|
:reconnect_interval,
|
|
73
79
|
:heartbeat_interval, :heartbeat_ttl, :heartbeat_timeout,
|
|
74
80
|
:max_message_size,
|
|
81
|
+
:sndbuf, :rcvbuf,
|
|
75
82
|
:on_mute,
|
|
76
83
|
:mechanism,
|
|
77
84
|
:qos
|
|
@@ -23,6 +23,7 @@ module OMQ
|
|
|
23
23
|
Routing.drain_send_queue(q, batch)
|
|
24
24
|
batch.each { |parts| conn.write_message(parts) }
|
|
25
25
|
conn.flush
|
|
26
|
+
batch.each { |parts| engine.emit_verbose_monitor_event(:message_sent, parts: parts) }
|
|
26
27
|
rescue Protocol::ZMTP::Error, *CONNECTION_LOST
|
|
27
28
|
engine.connection_lost(conn)
|
|
28
29
|
break
|
data/lib/omq/routing/dealer.rb
CHANGED
|
@@ -16,6 +16,7 @@ module OMQ
|
|
|
16
16
|
#
|
|
17
17
|
def initialize
|
|
18
18
|
@queues = [] # ordered list of per-connection inner queues
|
|
19
|
+
@drain = [] # orphaned queues, drained before active queues
|
|
19
20
|
@mapping = {} # connection => inner queue
|
|
20
21
|
@cycle = @queues.cycle # live reference — sees adds/removes
|
|
21
22
|
@condition = Async::Condition.new
|
|
@@ -37,18 +38,18 @@ module OMQ
|
|
|
37
38
|
|
|
38
39
|
# Removes the per-connection queue for a disconnected peer.
|
|
39
40
|
#
|
|
40
|
-
# If the queue
|
|
41
|
-
#
|
|
42
|
-
#
|
|
43
|
-
#
|
|
41
|
+
# If the queue still has pending messages it moves to the
|
|
42
|
+
# priority drain list so those messages are consumed before
|
|
43
|
+
# any active connection's messages — preserving FIFO for
|
|
44
|
+
# sequential connections.
|
|
44
45
|
#
|
|
45
46
|
# @param conn [Connection]
|
|
46
47
|
#
|
|
47
48
|
def remove_queue(conn)
|
|
48
49
|
q = @mapping.delete(conn)
|
|
49
50
|
return unless q
|
|
50
|
-
@queues.delete(q)
|
|
51
|
-
|
|
51
|
+
@queues.delete(q)
|
|
52
|
+
@drain << q unless q.empty?
|
|
52
53
|
end
|
|
53
54
|
|
|
54
55
|
|
|
@@ -73,7 +74,7 @@ module OMQ
|
|
|
73
74
|
return try_dequeue if timeout == 0
|
|
74
75
|
|
|
75
76
|
loop do
|
|
76
|
-
return nil if @closed && @queues.all?(&:empty?)
|
|
77
|
+
return nil if @closed && @drain.empty? && @queues.all?(&:empty?)
|
|
77
78
|
|
|
78
79
|
msg = try_dequeue
|
|
79
80
|
return msg if msg
|
|
@@ -100,17 +101,25 @@ module OMQ
|
|
|
100
101
|
# @return [Boolean]
|
|
101
102
|
#
|
|
102
103
|
def empty?
|
|
103
|
-
@queues.all?(&:empty?)
|
|
104
|
+
@drain.empty? && @queues.all?(&:empty?)
|
|
104
105
|
end
|
|
105
106
|
|
|
106
107
|
private
|
|
107
108
|
|
|
108
|
-
#
|
|
109
|
-
#
|
|
110
|
-
# Lazily removes empty orphaned queues (disconnected peers that have
|
|
111
|
-
# been fully drained).
|
|
109
|
+
# Drains orphaned queues first (preserves FIFO for disconnected
|
|
110
|
+
# peers), then tries each active queue once in round-robin order.
|
|
112
111
|
#
|
|
113
112
|
def try_dequeue
|
|
113
|
+
# Priority: drain orphaned queues before serving active ones
|
|
114
|
+
until @drain.empty?
|
|
115
|
+
msg = @drain.first.dequeue(timeout: 0)
|
|
116
|
+
if msg
|
|
117
|
+
return msg
|
|
118
|
+
else
|
|
119
|
+
@drain.shift
|
|
120
|
+
end
|
|
121
|
+
end
|
|
122
|
+
|
|
114
123
|
@queues.size.times do
|
|
115
124
|
q = begin
|
|
116
125
|
@cycle.next
|
|
@@ -120,10 +129,6 @@ module OMQ
|
|
|
120
129
|
end
|
|
121
130
|
msg = q.dequeue(timeout: 0)
|
|
122
131
|
return msg if msg
|
|
123
|
-
if q.empty? && !@mapping.value?(q)
|
|
124
|
-
@queues.delete(q)
|
|
125
|
-
break
|
|
126
|
-
end
|
|
127
132
|
end
|
|
128
133
|
nil
|
|
129
134
|
end
|
data/lib/omq/routing/fan_out.rb
CHANGED
|
@@ -162,7 +162,10 @@ module OMQ
|
|
|
162
162
|
loop do
|
|
163
163
|
batch = [q.dequeue]
|
|
164
164
|
Routing.drain_send_queue(q, batch)
|
|
165
|
-
|
|
165
|
+
if write_matching_batch(conn, batch, use_wire)
|
|
166
|
+
conn.flush
|
|
167
|
+
batch.each { |parts| @engine.emit_verbose_monitor_event(:message_sent, parts: parts) }
|
|
168
|
+
end
|
|
166
169
|
rescue Protocol::ZMTP::Error, *CONNECTION_LOST
|
|
167
170
|
@engine.connection_lost(conn)
|
|
168
171
|
break
|
|
@@ -193,6 +196,7 @@ module OMQ
|
|
|
193
196
|
begin
|
|
194
197
|
conn.write_message(latest)
|
|
195
198
|
conn.flush
|
|
199
|
+
@engine.emit_verbose_monitor_event(:message_sent, parts: latest)
|
|
196
200
|
rescue Protocol::ZMTP::Error, *CONNECTION_LOST
|
|
197
201
|
@engine.connection_lost(conn)
|
|
198
202
|
break
|
data/lib/omq/routing/pair.rb
CHANGED
|
@@ -18,7 +18,7 @@ module OMQ
|
|
|
18
18
|
@connection = nil
|
|
19
19
|
@recv_queue = FairQueue.new
|
|
20
20
|
@send_queue = nil # created per-connection
|
|
21
|
-
@staging_queue =
|
|
21
|
+
@staging_queue = StagingQueue.new(@engine.options.send_hwm)
|
|
22
22
|
@send_pump = nil
|
|
23
23
|
@tasks = []
|
|
24
24
|
end
|
|
@@ -39,7 +39,7 @@ module OMQ
|
|
|
39
39
|
|
|
40
40
|
unless connection.is_a?(Transport::Inproc::DirectPipe)
|
|
41
41
|
@send_queue = Routing.build_queue(@engine.options.send_hwm, :block)
|
|
42
|
-
while (msg = @staging_queue.dequeue
|
|
42
|
+
while (msg = @staging_queue.dequeue)
|
|
43
43
|
@send_queue.enqueue(msg)
|
|
44
44
|
end
|
|
45
45
|
start_send_pump(connection)
|
|
@@ -53,7 +53,12 @@ module OMQ
|
|
|
53
53
|
if @connection == connection
|
|
54
54
|
@connection = nil
|
|
55
55
|
@recv_queue.remove_queue(connection)
|
|
56
|
-
@send_queue
|
|
56
|
+
if @send_queue
|
|
57
|
+
while (msg = @send_queue.dequeue(timeout: 0))
|
|
58
|
+
@staging_queue.prepend(msg)
|
|
59
|
+
end
|
|
60
|
+
@send_queue = nil
|
|
61
|
+
end
|
|
57
62
|
@send_pump&.stop
|
|
58
63
|
@send_pump = nil
|
|
59
64
|
end
|
|
@@ -100,7 +105,9 @@ module OMQ
|
|
|
100
105
|
begin
|
|
101
106
|
batch.each { |parts| conn.write_message(parts) }
|
|
102
107
|
conn.flush
|
|
108
|
+
batch.each { |parts| @engine.emit_verbose_monitor_event(:message_sent, parts: parts) }
|
|
103
109
|
rescue Protocol::ZMTP::Error, *CONNECTION_LOST
|
|
110
|
+
batch.each { |parts| @staging_queue.prepend(parts) }
|
|
104
111
|
@engine.connection_lost(conn)
|
|
105
112
|
break
|
|
106
113
|
end
|
data/lib/omq/routing/push.rb
CHANGED
data/lib/omq/routing/req.rb
CHANGED
|
@@ -35,7 +35,7 @@ module OMQ
|
|
|
35
35
|
@conn_queues = {} # connection => send queue
|
|
36
36
|
@conn_send_tasks = {} # connection => send pump task
|
|
37
37
|
@direct_pipe = nil
|
|
38
|
-
@staging_queue =
|
|
38
|
+
@staging_queue = StagingQueue.new(@engine.options.send_hwm)
|
|
39
39
|
end
|
|
40
40
|
|
|
41
41
|
|
|
@@ -45,11 +45,18 @@ module OMQ
|
|
|
45
45
|
# @param conn [Connection]
|
|
46
46
|
#
|
|
47
47
|
def add_round_robin_send_connection(conn)
|
|
48
|
-
update_direct_pipe
|
|
49
48
|
q = Routing.build_queue(@engine.options.send_hwm, :block)
|
|
50
49
|
@conn_queues[conn] = q
|
|
51
|
-
drain_staging_to(q)
|
|
52
50
|
start_conn_send_pump(conn, q)
|
|
51
|
+
drain_staging_to(q)
|
|
52
|
+
@connections << conn
|
|
53
|
+
update_direct_pipe
|
|
54
|
+
# A message may have squeezed into staging while drain was
|
|
55
|
+
# running (the pipe loop's blocked enqueue completed after
|
|
56
|
+
# drain freed space). Now that @connections is set, no new
|
|
57
|
+
# messages will go to staging, so this second drain is
|
|
58
|
+
# guaranteed to finish quickly.
|
|
59
|
+
drain_staging_to(q)
|
|
53
60
|
signal_connection_available
|
|
54
61
|
end
|
|
55
62
|
|
|
@@ -61,8 +68,14 @@ module OMQ
|
|
|
61
68
|
#
|
|
62
69
|
def remove_round_robin_send_connection(conn)
|
|
63
70
|
update_direct_pipe
|
|
64
|
-
@conn_queues.delete(conn)
|
|
65
|
-
|
|
71
|
+
q = @conn_queues.delete(conn)
|
|
72
|
+
if q
|
|
73
|
+
while (msg = q.dequeue(timeout: 0))
|
|
74
|
+
@staging_queue.prepend(msg)
|
|
75
|
+
end
|
|
76
|
+
q.close
|
|
77
|
+
end
|
|
78
|
+
@conn_send_tasks.delete(conn)
|
|
66
79
|
end
|
|
67
80
|
|
|
68
81
|
|
|
@@ -105,6 +118,8 @@ module OMQ
|
|
|
105
118
|
conn = next_connection
|
|
106
119
|
@conn_queues[conn].enqueue(parts)
|
|
107
120
|
end
|
|
121
|
+
rescue Async::Queue::ClosedError
|
|
122
|
+
retry
|
|
108
123
|
end
|
|
109
124
|
|
|
110
125
|
|
|
@@ -113,9 +128,13 @@ module OMQ
|
|
|
113
128
|
# were enqueued before any connection existed.
|
|
114
129
|
#
|
|
115
130
|
def drain_staging_to(q)
|
|
116
|
-
while (msg = @staging_queue.dequeue
|
|
131
|
+
while (msg = @staging_queue.dequeue)
|
|
117
132
|
q.enqueue(msg)
|
|
118
133
|
end
|
|
134
|
+
rescue Async::Queue::ClosedError
|
|
135
|
+
# Connection dropped while draining — put the undelivered
|
|
136
|
+
# message back at the front so ordering is preserved.
|
|
137
|
+
@staging_queue.prepend(msg) if msg
|
|
119
138
|
end
|
|
120
139
|
|
|
121
140
|
|
|
@@ -127,8 +146,10 @@ module OMQ
|
|
|
127
146
|
def next_connection
|
|
128
147
|
@cycle.next
|
|
129
148
|
rescue StopIteration
|
|
130
|
-
@
|
|
131
|
-
|
|
149
|
+
if @connections.empty?
|
|
150
|
+
@connection_available = Async::Promise.new
|
|
151
|
+
@connection_available.wait
|
|
152
|
+
end
|
|
132
153
|
@cycle = @connections.cycle
|
|
133
154
|
retry
|
|
134
155
|
end
|
|
@@ -153,10 +174,14 @@ module OMQ
|
|
|
153
174
|
def start_conn_send_pump(conn, q)
|
|
154
175
|
task = @engine.spawn_pump_task(annotation: "send pump") do
|
|
155
176
|
loop do
|
|
156
|
-
|
|
177
|
+
msg = q.dequeue
|
|
178
|
+
break unless msg # queue closed by remove_round_robin_send_connection
|
|
179
|
+
batch = [msg]
|
|
157
180
|
Routing.drain_send_queue(q, batch)
|
|
158
181
|
write_batch(conn, batch)
|
|
182
|
+
batch.each { |parts| @engine.emit_verbose_monitor_event(:message_sent, parts: parts) }
|
|
159
183
|
rescue Protocol::ZMTP::Error, *CONNECTION_LOST
|
|
184
|
+
batch&.each { |parts| @staging_queue.prepend(parts) }
|
|
160
185
|
@engine.connection_lost(conn)
|
|
161
186
|
break
|
|
162
187
|
end
|
|
@@ -0,0 +1,60 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
module OMQ
|
|
4
|
+
module Routing
|
|
5
|
+
# Bounded FIFO queue for staging unsent messages.
|
|
6
|
+
#
|
|
7
|
+
# Wraps an +Async::LimitedQueue+ for backpressure, with a small
|
|
8
|
+
# prepend buffer checked first on dequeue (same trick as the
|
|
9
|
+
# prefetch buffer in {OMQ::Readable#receive}).
|
|
10
|
+
#
|
|
11
|
+
class StagingQueue
|
|
12
|
+
# @param max [Integer, nil] capacity (nil or 0 = unbounded)
|
|
13
|
+
#
|
|
14
|
+
def initialize(max = nil)
|
|
15
|
+
@queue = (max && max > 0) ? Async::LimitedQueue.new(max) : Async::Queue.new
|
|
16
|
+
@head = []
|
|
17
|
+
@mu = Mutex.new
|
|
18
|
+
end
|
|
19
|
+
|
|
20
|
+
|
|
21
|
+
# Appends a message to the back.
|
|
22
|
+
# Blocks (fiber-yields) when at capacity.
|
|
23
|
+
#
|
|
24
|
+
# @param msg [Array<String>]
|
|
25
|
+
# @return [void]
|
|
26
|
+
#
|
|
27
|
+
def enqueue(msg)
|
|
28
|
+
@queue.enqueue(msg)
|
|
29
|
+
end
|
|
30
|
+
|
|
31
|
+
|
|
32
|
+
# Inserts a message at the front (for re-staging after a
|
|
33
|
+
# failed drain).
|
|
34
|
+
#
|
|
35
|
+
# @param msg [Array<String>]
|
|
36
|
+
# @return [void]
|
|
37
|
+
#
|
|
38
|
+
def prepend(msg)
|
|
39
|
+
@mu.synchronize { @head.push(msg) }
|
|
40
|
+
end
|
|
41
|
+
|
|
42
|
+
|
|
43
|
+
# Returns the first message: from the prepend buffer if
|
|
44
|
+
# non-empty, otherwise non-blocking dequeue from the main queue.
|
|
45
|
+
#
|
|
46
|
+
# @return [Array<String>, nil]
|
|
47
|
+
#
|
|
48
|
+
def dequeue
|
|
49
|
+
@mu.synchronize { @head.shift } || @queue.dequeue(timeout: 0)
|
|
50
|
+
end
|
|
51
|
+
|
|
52
|
+
|
|
53
|
+
# @return [Boolean]
|
|
54
|
+
#
|
|
55
|
+
def empty?
|
|
56
|
+
@mu.synchronize { @head.empty? } && @queue.empty?
|
|
57
|
+
end
|
|
58
|
+
end
|
|
59
|
+
end
|
|
60
|
+
end
|
data/lib/omq/routing.rb
CHANGED
|
@@ -4,6 +4,7 @@ require "async"
|
|
|
4
4
|
require "async/queue"
|
|
5
5
|
require "async/limited_queue"
|
|
6
6
|
require_relative "drop_queue"
|
|
7
|
+
require_relative "routing/staging_queue"
|
|
7
8
|
require_relative "routing/fair_queue"
|
|
8
9
|
require_relative "routing/fair_recv"
|
|
9
10
|
require_relative "routing/conn_send_pump"
|
|
@@ -0,0 +1,23 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
module OMQ
|
|
4
|
+
# Mixin that rejects multipart messages.
|
|
5
|
+
#
|
|
6
|
+
# All draft socket types (CLIENT, SERVER, RADIO, DISH, SCATTER,
|
|
7
|
+
# GATHER, PEER, CHANNEL) require single-frame messages for
|
|
8
|
+
# thread-safe atomic operations.
|
|
9
|
+
#
|
|
10
|
+
module SingleFrame
|
|
11
|
+
# Sends a message, rejecting multipart messages.
|
|
12
|
+
#
|
|
13
|
+
# @param message [String, Array<String>] message to send (must be single-frame)
|
|
14
|
+
# @raise [ArgumentError] if a multipart message is provided
|
|
15
|
+
# @return [void]
|
|
16
|
+
def send(message)
|
|
17
|
+
if message.is_a?(Array) && message.size > 1
|
|
18
|
+
raise ArgumentError, "#{self.class} does not support multipart messages"
|
|
19
|
+
end
|
|
20
|
+
super
|
|
21
|
+
end
|
|
22
|
+
end
|
|
23
|
+
end
|
data/lib/omq/socket.rb
CHANGED
|
@@ -36,6 +36,8 @@ module OMQ
|
|
|
36
36
|
:heartbeat_ttl, :heartbeat_ttl=,
|
|
37
37
|
:heartbeat_timeout, :heartbeat_timeout=,
|
|
38
38
|
:max_message_size, :max_message_size=,
|
|
39
|
+
:sndbuf, :sndbuf=,
|
|
40
|
+
:rcvbuf, :rcvbuf=,
|
|
39
41
|
:on_mute, :on_mute=,
|
|
40
42
|
:mechanism, :mechanism=
|
|
41
43
|
|
|
@@ -170,10 +172,11 @@ module OMQ
|
|
|
170
172
|
# # later:
|
|
171
173
|
# task.stop
|
|
172
174
|
#
|
|
173
|
-
def monitor(&block)
|
|
175
|
+
def monitor(verbose: false, &block)
|
|
174
176
|
ensure_parent_task
|
|
175
177
|
queue = Async::Queue.new
|
|
176
178
|
@engine.monitor_queue = queue
|
|
179
|
+
@engine.verbose_monitor = verbose
|
|
177
180
|
Reactor.run do
|
|
178
181
|
@engine.parent_task.async(transient: true, annotation: "monitor") do
|
|
179
182
|
while (event = queue.dequeue)
|
data/lib/omq/transport/ipc.rb
CHANGED
|
@@ -27,7 +27,7 @@ module OMQ
|
|
|
27
27
|
|
|
28
28
|
server = UNIXServer.new(sock_path)
|
|
29
29
|
|
|
30
|
-
Listener.new(endpoint, server, path)
|
|
30
|
+
Listener.new(endpoint, server, path, engine)
|
|
31
31
|
end
|
|
32
32
|
|
|
33
33
|
|
|
@@ -41,9 +41,15 @@ module OMQ
|
|
|
41
41
|
path = parse_path(endpoint)
|
|
42
42
|
sock_path = to_socket_path(path)
|
|
43
43
|
sock = UNIXSocket.new(sock_path)
|
|
44
|
+
apply_buffer_sizes(sock, engine.options)
|
|
44
45
|
engine.handle_connected(IO::Stream::Buffered.wrap(sock), endpoint: endpoint)
|
|
45
46
|
end
|
|
46
47
|
|
|
48
|
+
def apply_buffer_sizes(sock, options)
|
|
49
|
+
sock.setsockopt(Socket::SOL_SOCKET, Socket::SO_SNDBUF, options.sndbuf) if options.sndbuf
|
|
50
|
+
sock.setsockopt(Socket::SOL_SOCKET, Socket::SO_RCVBUF, options.rcvbuf) if options.rcvbuf
|
|
51
|
+
end
|
|
52
|
+
|
|
47
53
|
private
|
|
48
54
|
|
|
49
55
|
# Extracts path from "ipc://path".
|
|
@@ -87,11 +93,13 @@ module OMQ
|
|
|
87
93
|
# @param endpoint [String] the IPC endpoint URI
|
|
88
94
|
# @param server [UNIXServer]
|
|
89
95
|
# @param path [String] filesystem or abstract namespace path
|
|
96
|
+
# @param engine [Engine]
|
|
90
97
|
#
|
|
91
|
-
def initialize(endpoint, server, path)
|
|
98
|
+
def initialize(endpoint, server, path, engine)
|
|
92
99
|
@endpoint = endpoint
|
|
93
100
|
@server = server
|
|
94
101
|
@path = path
|
|
102
|
+
@engine = engine
|
|
95
103
|
@task = nil
|
|
96
104
|
end
|
|
97
105
|
|
|
@@ -106,11 +114,14 @@ module OMQ
|
|
|
106
114
|
@task = parent_task.async(transient: true, annotation: "ipc accept #{@endpoint}") do
|
|
107
115
|
loop do
|
|
108
116
|
client = @server.accept
|
|
117
|
+
IPC.apply_buffer_sizes(client, @engine.options)
|
|
109
118
|
Async::Task.current.defer_stop { on_accepted.call(IO::Stream::Buffered.wrap(client)) }
|
|
110
119
|
end
|
|
111
120
|
rescue Async::Stop
|
|
112
121
|
rescue IOError
|
|
113
122
|
# server closed
|
|
123
|
+
rescue => e
|
|
124
|
+
$stderr.write("omq: ipc accept: #{e.class}: #{e.message}\n#{e.backtrace&.first}\n") if OMQ::DEBUG
|
|
114
125
|
ensure
|
|
115
126
|
@server.close rescue nil
|
|
116
127
|
end
|
data/lib/omq/transport/tcp.rb
CHANGED
|
@@ -34,7 +34,7 @@ module OMQ
|
|
|
34
34
|
|
|
35
35
|
host_part = host.include?(":") ? "[#{host}]" : host
|
|
36
36
|
resolved = "tcp://#{host_part}:#{actual_port}"
|
|
37
|
-
Listener.new(resolved, servers, actual_port)
|
|
37
|
+
Listener.new(resolved, servers, actual_port, engine)
|
|
38
38
|
end
|
|
39
39
|
|
|
40
40
|
|
|
@@ -58,6 +58,7 @@ module OMQ
|
|
|
58
58
|
def connect(endpoint, engine)
|
|
59
59
|
host, port = self.parse_endpoint(endpoint)
|
|
60
60
|
sock = TCPSocket.new(host, port)
|
|
61
|
+
apply_buffer_sizes(sock, engine.options)
|
|
61
62
|
engine.handle_connected(IO::Stream::Buffered.wrap(sock), endpoint: endpoint)
|
|
62
63
|
end
|
|
63
64
|
|
|
@@ -71,6 +72,12 @@ module OMQ
|
|
|
71
72
|
uri = URI.parse(endpoint)
|
|
72
73
|
[uri.hostname, uri.port]
|
|
73
74
|
end
|
|
75
|
+
|
|
76
|
+
|
|
77
|
+
def apply_buffer_sizes(sock, options)
|
|
78
|
+
sock.setsockopt(Socket::SOL_SOCKET, Socket::SO_SNDBUF, options.sndbuf) if options.sndbuf
|
|
79
|
+
sock.setsockopt(Socket::SOL_SOCKET, Socket::SO_RCVBUF, options.rcvbuf) if options.rcvbuf
|
|
80
|
+
end
|
|
74
81
|
end
|
|
75
82
|
|
|
76
83
|
|
|
@@ -93,11 +100,13 @@ module OMQ
|
|
|
93
100
|
# @param endpoint [String] resolved endpoint URI
|
|
94
101
|
# @param servers [Array<TCPServer>]
|
|
95
102
|
# @param port [Integer] bound port number
|
|
103
|
+
# @param engine [Engine]
|
|
96
104
|
#
|
|
97
|
-
def initialize(endpoint, servers, port)
|
|
105
|
+
def initialize(endpoint, servers, port, engine)
|
|
98
106
|
@endpoint = endpoint
|
|
99
107
|
@servers = servers
|
|
100
108
|
@port = port
|
|
109
|
+
@engine = engine
|
|
101
110
|
@tasks = []
|
|
102
111
|
end
|
|
103
112
|
|
|
@@ -113,11 +122,14 @@ module OMQ
|
|
|
113
122
|
parent_task.async(transient: true, annotation: "tcp accept #{@endpoint}") do
|
|
114
123
|
loop do
|
|
115
124
|
client = server.accept
|
|
125
|
+
TCP.apply_buffer_sizes(client, @engine.options)
|
|
116
126
|
Async::Task.current.defer_stop { on_accepted.call(IO::Stream::Buffered.wrap(client)) }
|
|
117
127
|
end
|
|
118
128
|
rescue Async::Stop
|
|
119
129
|
rescue IOError
|
|
120
130
|
# server closed
|
|
131
|
+
rescue => e
|
|
132
|
+
$stderr.write("omq: tcp accept: #{e.class}: #{e.message}\n#{e.backtrace&.first}\n") if OMQ::DEBUG
|
|
121
133
|
ensure
|
|
122
134
|
server.close rescue nil
|
|
123
135
|
end
|
data/lib/omq/version.rb
CHANGED
data/lib/omq.rb
CHANGED
|
@@ -13,6 +13,11 @@ require_relative "omq/version"
|
|
|
13
13
|
require_relative "omq/monitor_event"
|
|
14
14
|
|
|
15
15
|
module OMQ
|
|
16
|
+
# When OMQ_DEBUG is set, silent rescue clauses print the exception
|
|
17
|
+
# to stderr so transport/engine bugs surface immediately.
|
|
18
|
+
DEBUG = !!ENV["OMQ_DEBUG"]
|
|
19
|
+
|
|
20
|
+
|
|
16
21
|
# Raised when an internal pump task crashes unexpectedly.
|
|
17
22
|
# The socket is no longer usable; the original error is available via #cause.
|
|
18
23
|
#
|
|
@@ -71,6 +76,7 @@ require_relative "omq/engine"
|
|
|
71
76
|
require_relative "omq/queue_interface"
|
|
72
77
|
require_relative "omq/readable"
|
|
73
78
|
require_relative "omq/writable"
|
|
79
|
+
require_relative "omq/single_frame"
|
|
74
80
|
|
|
75
81
|
# Socket types
|
|
76
82
|
require_relative "omq/socket"
|
metadata
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
|
2
2
|
name: omq
|
|
3
3
|
version: !ruby/object:Gem::Version
|
|
4
|
-
version: 0.
|
|
4
|
+
version: 0.15.0
|
|
5
5
|
platform: ruby
|
|
6
6
|
authors:
|
|
7
7
|
- Patrik Wenger
|
|
@@ -96,9 +96,11 @@ files:
|
|
|
96
96
|
- lib/omq/routing/req.rb
|
|
97
97
|
- lib/omq/routing/round_robin.rb
|
|
98
98
|
- lib/omq/routing/router.rb
|
|
99
|
+
- lib/omq/routing/staging_queue.rb
|
|
99
100
|
- lib/omq/routing/sub.rb
|
|
100
101
|
- lib/omq/routing/xpub.rb
|
|
101
102
|
- lib/omq/routing/xsub.rb
|
|
103
|
+
- lib/omq/single_frame.rb
|
|
102
104
|
- lib/omq/socket.rb
|
|
103
105
|
- lib/omq/transport/inproc.rb
|
|
104
106
|
- lib/omq/transport/inproc/direct_pipe.rb
|