omq 0.19.3 → 0.21.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/CHANGELOG.md +82 -0
- data/README.md +75 -44
- data/lib/omq/engine/recv_pump.rb +35 -8
- data/lib/omq/engine.rb +5 -1
- data/lib/omq/options.rb +6 -3
- data/lib/omq/pair.rb +16 -3
- data/lib/omq/pub_sub.rb +53 -14
- data/lib/omq/push_pull.rb +11 -6
- data/lib/omq/req_rep.rb +31 -6
- data/lib/omq/router_dealer.rb +31 -6
- data/lib/omq/routing/dealer.rb +22 -5
- data/lib/omq/routing/pair.rb +24 -9
- data/lib/omq/routing/pull.rb +25 -8
- data/lib/omq/routing/rep.rb +22 -6
- data/lib/omq/routing/req.rb +23 -5
- data/lib/omq/routing/round_robin.rb +16 -3
- data/lib/omq/routing/router.rb +22 -7
- data/lib/omq/routing/sub.rb +13 -9
- data/lib/omq/routing/xpub.rb +3 -2
- data/lib/omq/routing/xsub.rb +11 -9
- data/lib/omq/routing.rb +0 -2
- data/lib/omq/socket.rb +28 -5
- data/lib/omq/version.rb +1 -1
- metadata +1 -3
- data/lib/omq/routing/fair_queue.rb +0 -191
- data/lib/omq/routing/fair_recv.rb +0 -53
checksums.yaml
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
SHA256:
|
|
3
|
-
metadata.gz:
|
|
4
|
-
data.tar.gz:
|
|
3
|
+
metadata.gz: f24ac91fe456168b4d369f63506cf1d3d2b48488637ce767363aa17bb542d2b3
|
|
4
|
+
data.tar.gz: ca1b98ab4083ad90f7f483bc9df937656ae9862c9fd07800fc46315543e5fe75
|
|
5
5
|
SHA512:
|
|
6
|
-
metadata.gz:
|
|
7
|
-
data.tar.gz:
|
|
6
|
+
metadata.gz: 16f20fa600ebd66228589da4e03efc809a616d6a74fbe3460a8ced6811349fb86903e43a440708e1e78f102af711d6f9ea3ebf9cbc0e7fb4aef9591db5061e39
|
|
7
|
+
data.tar.gz: 89ff3da1bb5caa2223d1e323a389cacc4f270fed719902dbd0721c56da8ff5c0a1cbf1e6eebc59c8386f470c433c69dfb20462d46f84c9a2fa848b3e1364dfd0
|
data/CHANGELOG.md
CHANGED
|
@@ -1,5 +1,87 @@
|
|
|
1
1
|
# Changelog
|
|
2
2
|
|
|
3
|
+
## 0.21.0 — 2026-04-15
|
|
4
|
+
|
|
5
|
+
### Changed
|
|
6
|
+
|
|
7
|
+
- **Recv path: shared queue, no more `FairQueue`.** Every fair-queue
|
|
8
|
+
routing strategy (Pull, Pair, Rep, Dealer, Router, Req, Sub, XSub)
|
|
9
|
+
now owns a single `Async::LimitedQueue` sized to `recv_hwm`. Each
|
|
10
|
+
connection's recv pump writes directly into it. `FairQueue`,
|
|
11
|
+
`SignalingQueue`, and the `FairRecv` mixin are deleted. Cross-peer
|
|
12
|
+
fairness comes entirely from the pump yield limit; per-connection
|
|
13
|
+
ordering is preserved; cross-connection ordering was never a
|
|
14
|
+
guarantee. Symmetric with the send side, which already uses one
|
|
15
|
+
work-stealing queue per socket. Sister gems (channel, clientserver,
|
|
16
|
+
p2p, radiodish, scattergather, qos) updated to match.
|
|
17
|
+
- **Recv pump fairness bumped to 256 msgs / 512 KiB** (was 64 / 1 MiB),
|
|
18
|
+
symmetric with `RoundRobin::BATCH_MSG_CAP` / `BATCH_BYTE_CAP` on the
|
|
19
|
+
send side.
|
|
20
|
+
|
|
21
|
+
### Added
|
|
22
|
+
|
|
23
|
+
- **`Socket#attach_endpoints` accepts arrays.** Constructors passed an
|
|
24
|
+
array of endpoint strings now bind/connect each one in order, so
|
|
25
|
+
`OMQ::SUB.new(["inproc://a", "inproc://b"])` works.
|
|
26
|
+
- **PUB/SUB regression test** for a SUB with sequential post-hoc
|
|
27
|
+
`#connect` calls to multiple bound PUBs, mirroring the SCATTER/GATHER
|
|
28
|
+
post-hoc-connect coverage.
|
|
29
|
+
- **DESIGN.md: "Libzmq quirks OMQ avoids"** — per-pipe HWM (actual
|
|
30
|
+
buffering is `send_hwm × N_peers`, forces strict RR, slow-worker
|
|
31
|
+
stall footgun) and the edge-triggered `ZMQ_FD` that fires spuriously
|
|
32
|
+
and misses edges, requiring the `ZMQ_EVENTS` / `ZMQ_DONTWAIT` dance.
|
|
33
|
+
README "Socket Types" condensed to a pointer at DESIGN.md.
|
|
34
|
+
|
|
35
|
+
## 0.20.0 — 2026-04-14
|
|
36
|
+
|
|
37
|
+
### Changed
|
|
38
|
+
|
|
39
|
+
- **Default `linger` is now `Float::INFINITY`** (matches libzmq). Sockets
|
|
40
|
+
wait forever on close for queued messages to drain unless `linger` is
|
|
41
|
+
set explicitly. Pass `linger: 0` to keep the old "drop on close"
|
|
42
|
+
behavior. `Options#linger` now always returns a `Numeric` (never `nil`).
|
|
43
|
+
- **Socket constructors accept a block.** `OMQ::PUSH.new { |p| ... }`
|
|
44
|
+
yields the socket, then closes it (even on exception) — `File.open`
|
|
45
|
+
style. Applies to every socket type.
|
|
46
|
+
- **Per-socket-type constructors take the full kwarg set** they support:
|
|
47
|
+
`send_hwm`, `recv_hwm`, `send_timeout`, `recv_timeout`, `linger`,
|
|
48
|
+
`backend`, plus pattern-specific ones (`subscribe:`, `on_mute:`,
|
|
49
|
+
`conflate:`). Previously some only accepted `linger`.
|
|
50
|
+
- **Hot-path recv pump: size-1 fast path for byte counting.** The
|
|
51
|
+
`FAIRNESS_BYTES` accumulator in `RecvPump#start_direct` (and its
|
|
52
|
+
transform variant) now short-circuits single-frame messages instead
|
|
53
|
+
of iterating, keeping both entry methods monomorphic for YJIT.
|
|
54
|
+
- **Hot-path round-robin `batch_bytes`** short-circuits single-frame
|
|
55
|
+
batches the same way, replacing `parts.sum { ... }` with a direct
|
|
56
|
+
`bytesize` call.
|
|
57
|
+
- **Fair-queue single-connection fast path.** `try_dequeue` now skips
|
|
58
|
+
`Enumerator#next` when a fair-queue recv socket has exactly one peer
|
|
59
|
+
(the common case) and dequeues directly from the sole queue.
|
|
60
|
+
- **`drain_send_queues` is cancellation-safe.** `Async::Stop` raised at
|
|
61
|
+
the drain sleep point (e.g. from a parent `task.stop`) is now rescued
|
|
62
|
+
so `Socket#close` can finish the rest of its teardown instead of
|
|
63
|
+
propagating the cancellation out of the ensure path.
|
|
64
|
+
- **Hot-path `Array#[0]` → `Array#first`** in writable batching and
|
|
65
|
+
pair routing — `#first` has a dedicated YJIT specialization that is
|
|
66
|
+
measurably faster on single-frame messages.
|
|
67
|
+
- **Benchmark size sweep reworked.** `SIZES` is now a ×4 geometric
|
|
68
|
+
progression `128, 512, 2048, 8192, 32_768` bytes, replacing
|
|
69
|
+
`64 / 1024 / 8192 / 65_536`. Fills the 64 B → 1 KiB gap, drops 64 KiB
|
|
70
|
+
(tcp/ipc already saturated at 32 KiB, inproc regressed). `report.rb
|
|
71
|
+
--update-readme` and `bench/README.md` regenerated.
|
|
72
|
+
|
|
73
|
+
### Fixed
|
|
74
|
+
|
|
75
|
+
- **Slow `send_timeout` test.** The `raises IO::TimeoutError when send
|
|
76
|
+
blocks longer than send_timeout` test now constructs its PUSH with
|
|
77
|
+
`linger: 0`. Previously the undeliverable fill message combined with
|
|
78
|
+
the new default `linger: Float::INFINITY` made the close-in-ensure
|
|
79
|
+
path wait out the full linger budget, silently eating the enclosing
|
|
80
|
+
`task.with_timeout` and inflating suite runtime.
|
|
81
|
+
- **Test suite runtime.** `TEST_ASYNC_TIMEOUT` lowered from 5 s to 1 s:
|
|
82
|
+
real hangs fail fast and the full suite finishes in ~3 s instead of
|
|
83
|
+
~8 s.
|
|
84
|
+
|
|
3
85
|
## 0.19.3 — 2026-04-13
|
|
4
86
|
|
|
5
87
|
### Changed
|
data/README.md
CHANGED
|
@@ -1,42 +1,49 @@
|
|
|
1
|
-
#
|
|
1
|
+
# ØMQ — ZeroMQ for Ruby, no C required
|
|
2
2
|
|
|
3
3
|
[](https://github.com/zeromq/omq/actions/workflows/ci.yml)
|
|
4
4
|
[](https://rubygems.org/gems/omq)
|
|
5
5
|
[](LICENSE)
|
|
6
6
|
[](https://www.ruby-lang.org)
|
|
7
7
|
|
|
8
|
-
|
|
9
|
-
|
|
10
|
-
OMQ builds ZeroMQ socket patterns on top of [protocol-zmtp](https://github.com/paddor/protocol-zmtp) (a pure Ruby [ZMTP 3.1](https://rfc.zeromq.org/spec/23/) codec) using [Async](https://github.com/socketry/async) fibers. It speaks native ZeroMQ on the wire and interoperates with libzmq, pyzmq, CZMQ, and everything else in the ZMQ ecosystem.
|
|
11
|
-
|
|
12
|
-
> **980k msg/s** inproc | **38k msg/s** ipc | **31k msg/s** tcp
|
|
8
|
+
> **1.23M msg/s** inproc | **361k msg/s** ipc | **358k msg/s** tcp
|
|
13
9
|
>
|
|
14
|
-
> **
|
|
10
|
+
> **9.1 µs** inproc latency | **49 µs** ipc | **64 µs** tcp
|
|
15
11
|
>
|
|
16
12
|
> Ruby 4.0 + YJIT on a Linux VM — see [`bench/`](bench/) for full results
|
|
17
13
|
|
|
18
|
-
|
|
19
|
-
|
|
20
|
-
## What is ZeroMQ?
|
|
21
|
-
|
|
22
|
-
Brokerless message-oriented middleware. No central server, no extra hop — processes talk directly to each other, cutting latency in half compared to broker-based systems. You get the patterns you'd normally build on top of RabbitMQ or Redis — pub/sub, work distribution, request/reply, fan-out — but decentralized, with no single point of failure.
|
|
23
|
-
|
|
24
|
-
Networking is hard. ZeroMQ abstracts away reconnection, queuing, load balancing, and framing so you can focus on what your system actually does. Start with threads talking over `inproc://`, split into processes with `ipc://`, scale across machines with `tcp://` — same code, same API, just change the URL.
|
|
14
|
+
`gem install omq` and you're done. No libzmq, no compiler, no system packages —
|
|
15
|
+
just Ruby talking to every other ZeroMQ peer out there.
|
|
25
16
|
|
|
26
|
-
|
|
17
|
+
ØMQ gives your Ruby processes a way to talk to each other — and to anything
|
|
18
|
+
else speaking ZeroMQ — without a broker in the middle. Same API whether they
|
|
19
|
+
live in the same process, on the same machine, or across the network.
|
|
20
|
+
Reconnects, queuing, and back-pressure are handled for you; you write the
|
|
21
|
+
interesting part.
|
|
27
22
|
|
|
28
|
-
|
|
23
|
+
New to ZeroMQ? Start with [GETTING_STARTED.md](GETTING_STARTED.md) — a ~30 min
|
|
24
|
+
walkthrough of every major pattern with working code.
|
|
29
25
|
|
|
30
26
|
## Highlights
|
|
31
27
|
|
|
32
|
-
- **Zero dependencies on C** — no extensions, no FFI, no libzmq. `gem install`
|
|
33
|
-
|
|
34
|
-
- **
|
|
35
|
-
|
|
36
|
-
-
|
|
37
|
-
- **
|
|
38
|
-
|
|
39
|
-
- **
|
|
28
|
+
- **Zero dependencies on C** — no extensions, no FFI, no libzmq. `gem install`
|
|
29
|
+
just works everywhere
|
|
30
|
+
- **Fast** — YJIT-optimized hot paths, batched sends, GC-tuned allocations,
|
|
31
|
+
buffered I/O via [io-stream](https://github.com/socketry/io-stream),
|
|
32
|
+
direct-pipe inproc bypass
|
|
33
|
+
- **[`omq` CLI](https://github.com/paddor/omq-cli)** — a powerful swiss army
|
|
34
|
+
knife for ØMQ. `gem install omq-cli`
|
|
35
|
+
- **Every socket pattern** — req/rep, pub/sub, push/pull, dealer/router,
|
|
36
|
+
xpub/xsub, pair, and all draft types
|
|
37
|
+
- **Every transport** — tcp, ipc (Unix domain sockets), inproc (in-process
|
|
38
|
+
queues)
|
|
39
|
+
- **Async-native** — built on fibers, non-blocking from the ground up
|
|
40
|
+
- **Works outside Async too** — a shared IO thread handles sockets for callers
|
|
41
|
+
that aren't inside a reactor, so simple scripts just work
|
|
42
|
+
- **Wire-compatible** — interoperates with libzmq, pyzmq, CZMQ, zmq.rs over tcp
|
|
43
|
+
and ipc
|
|
44
|
+
- **Bind/connect order doesn't matter** — connect before bind, bind before
|
|
45
|
+
connect, peers come and go. ZeroMQ reconnects automatically and queued
|
|
46
|
+
messages drain when peers arrive
|
|
40
47
|
|
|
41
48
|
For architecture internals, see [DESIGN.md](DESIGN.md).
|
|
42
49
|
|
|
@@ -108,7 +115,8 @@ end
|
|
|
108
115
|
|
|
109
116
|
### Without Async (IO thread)
|
|
110
117
|
|
|
111
|
-
OMQ spawns a shared `omq-io` thread when used outside an Async reactor — no
|
|
118
|
+
OMQ spawns a shared `omq-io` thread when used outside an Async reactor — no
|
|
119
|
+
boilerplate needed:
|
|
112
120
|
|
|
113
121
|
```ruby
|
|
114
122
|
require 'omq'
|
|
@@ -123,7 +131,8 @@ push.close
|
|
|
123
131
|
pull.close
|
|
124
132
|
```
|
|
125
133
|
|
|
126
|
-
The IO thread runs all pumps, reconnection, and heartbeating in the background.
|
|
134
|
+
The IO thread runs all pumps, reconnection, and heartbeating in the background.
|
|
135
|
+
When you're inside an `Async` block, OMQ uses the existing reactor instead.
|
|
127
136
|
|
|
128
137
|
### Queue Interface
|
|
129
138
|
|
|
@@ -147,7 +156,11 @@ end
|
|
|
147
156
|
|
|
148
157
|
## Socket Types
|
|
149
158
|
|
|
150
|
-
All sockets are thread-safe. Default HWM is 1000 messages per socket.
|
|
159
|
+
All sockets are thread-safe. Default HWM is 1000 messages per socket.
|
|
160
|
+
`max_message_size` defaults to **`nil` (unlimited)** — set
|
|
161
|
+
`socket.max_message_size = N` to cap inbound frames at `N` bytes; oversized
|
|
162
|
+
frames cause the connection to be dropped before the body is read from the
|
|
163
|
+
wire. Classes live under `OMQ::` (alias: `ØMQ`).
|
|
151
164
|
|
|
152
165
|
#### Standard (multipart messages)
|
|
153
166
|
|
|
@@ -160,23 +173,28 @@ All sockets are thread-safe. Default HWM is 1000 messages per socket. `max_messa
|
|
|
160
173
|
| **XPUB** / **XSUB** | Fan-out (subscription events) | Fair-queue | Drop |
|
|
161
174
|
| **PAIR** | Exclusive 1-to-1 | Exclusive 1-to-1 | Block |
|
|
162
175
|
|
|
163
|
-
> **Work-stealing
|
|
176
|
+
> **Work-stealing, not round-robin.** Outbound load balancing uses one shared
|
|
177
|
+
> send queue per socket drained by N racing pump fibers, so a slow peer can't
|
|
178
|
+
> stall the pipeline. Under tight bursts on small `n`, distribution isn't
|
|
179
|
+
> strict RR. See [DESIGN.md](DESIGN.md#per-socket-hwm-not-per-connection) and
|
|
180
|
+
> [Libzmq quirks](DESIGN.md#libzmq-quirks-omq-avoids) for the reasoning.
|
|
164
181
|
|
|
165
182
|
#### Draft (single-frame only)
|
|
166
183
|
|
|
167
|
-
|
|
184
|
+
Each draft pattern lives in its own gem — install only the ones you use.
|
|
168
185
|
|
|
169
|
-
| Pattern | Send | Receive | When HWM full |
|
|
170
|
-
|
|
171
|
-
| **CLIENT** / **SERVER** | Work-stealing / routing-ID | Fair-queue | Block |
|
|
172
|
-
| **RADIO** / **DISH** | Group fan-out | Group filter | Drop |
|
|
173
|
-
| **SCATTER** / **GATHER** | Work-stealing | Fair-queue | Block |
|
|
174
|
-
| **PEER** | Routing-ID | Fair-queue | Block |
|
|
175
|
-
| **CHANNEL** | Exclusive 1-to-1 | Exclusive 1-to-1 | Block |
|
|
186
|
+
| Pattern | Send | Receive | When HWM full | Gem |
|
|
187
|
+
|---------|------|---------|---------------|-----|
|
|
188
|
+
| **CLIENT** / **SERVER** | Work-stealing / routing-ID | Fair-queue | Block | [`omq-rfc-clientserver`](https://github.com/paddor/omq-rfc-clientserver) |
|
|
189
|
+
| **RADIO** / **DISH** | Group fan-out | Group filter | Drop | [`omq-rfc-radiodish`](https://github.com/paddor/omq-rfc-radiodish) |
|
|
190
|
+
| **SCATTER** / **GATHER** | Work-stealing | Fair-queue | Block | [`omq-rfc-scattergather`](https://github.com/paddor/omq-rfc-scattergather) |
|
|
191
|
+
| **PEER** | Routing-ID | Fair-queue | Block | [`omq-rfc-p2p`](https://github.com/paddor/omq-rfc-p2p) |
|
|
192
|
+
| **CHANNEL** | Exclusive 1-to-1 | Exclusive 1-to-1 | Block | [`omq-rfc-channel`](https://github.com/paddor/omq-rfc-channel) |
|
|
176
193
|
|
|
177
194
|
## CLI
|
|
178
195
|
|
|
179
|
-
Install [omq-cli](https://github.com/paddor/omq-cli) for a command-line tool
|
|
196
|
+
Install [omq-cli](https://github.com/paddor/omq-cli) for a command-line tool
|
|
197
|
+
that sends, receives, pipes, and transforms ZeroMQ messages from the terminal:
|
|
180
198
|
|
|
181
199
|
```sh
|
|
182
200
|
gem install omq-cli
|
|
@@ -189,8 +207,21 @@ See the [omq-cli README](https://github.com/paddor/omq-cli) for full documentati
|
|
|
189
207
|
|
|
190
208
|
## Companion Gems
|
|
191
209
|
|
|
192
|
-
- **[omq-ffi](https://github.com/paddor/omq-ffi)** — libzmq FFI backend. Same
|
|
193
|
-
|
|
210
|
+
- **[omq-ffi](https://github.com/paddor/omq-ffi)** — libzmq FFI backend. Same
|
|
211
|
+
OMQ socket API, but backed by libzmq instead of the pure Ruby ZMTP stack.
|
|
212
|
+
Useful for interop testing and when you need libzmq-specific features.
|
|
213
|
+
Requires libzmq installed.
|
|
214
|
+
- **[omq-ractor](https://github.com/paddor/omq-ractor)** — bridge OMQ sockets
|
|
215
|
+
into Ruby Ractors for true parallel processing across cores. I/O stays on the
|
|
216
|
+
main Ractor, worker Ractors do pure computation.
|
|
217
|
+
|
|
218
|
+
### Protocol extensions (RFCs)
|
|
219
|
+
|
|
220
|
+
Optional plug-ins that extend the ZMTP wire protocol. Each is a separate gem;
|
|
221
|
+
load the ones you need.
|
|
222
|
+
|
|
223
|
+
- **[omq-rfc-zstd](https://github.com/paddor/omq-rfc-zstd)** — transparent
|
|
224
|
+
Zstandard compression on the wire, negotiated per peer via READY properties.
|
|
194
225
|
|
|
195
226
|
## Development
|
|
196
227
|
|
|
@@ -210,16 +241,16 @@ the stack.
|
|
|
210
241
|
# clone OMQ and its sibling repos into the same parent directory
|
|
211
242
|
git clone https://github.com/paddor/omq.git
|
|
212
243
|
git clone https://github.com/paddor/protocol-zmtp.git
|
|
213
|
-
git clone https://github.com/paddor/
|
|
214
|
-
git clone https://github.com/paddor/omq-rfc-blake3zmq.git
|
|
215
|
-
git clone https://github.com/paddor/omq-rfc-channel.git
|
|
244
|
+
git clone https://github.com/paddor/omq-rfc-zstd.git
|
|
216
245
|
git clone https://github.com/paddor/omq-rfc-clientserver.git
|
|
217
|
-
git clone https://github.com/paddor/omq-rfc-p2p.git
|
|
218
|
-
git clone https://github.com/paddor/omq-rfc-qos.git
|
|
219
246
|
git clone https://github.com/paddor/omq-rfc-radiodish.git
|
|
220
247
|
git clone https://github.com/paddor/omq-rfc-scattergather.git
|
|
248
|
+
git clone https://github.com/paddor/omq-rfc-channel.git
|
|
249
|
+
git clone https://github.com/paddor/omq-rfc-p2p.git
|
|
250
|
+
git clone https://github.com/paddor/omq-rfc-qos.git
|
|
221
251
|
git clone https://github.com/paddor/omq-ffi.git
|
|
222
252
|
git clone https://github.com/paddor/omq-ractor.git
|
|
253
|
+
git clone https://github.com/paddor/nuckle.git
|
|
223
254
|
|
|
224
255
|
cd omq
|
|
225
256
|
OMQ_DEV=1 bundle install
|
data/lib/omq/engine/recv_pump.rb
CHANGED
|
@@ -15,22 +15,22 @@ module OMQ
|
|
|
15
15
|
class RecvPump
|
|
16
16
|
# Max messages read from one connection before yielding to the
|
|
17
17
|
# scheduler. Prevents a busy peer from starving its siblings in
|
|
18
|
-
# fair-queue recv sockets.
|
|
19
|
-
FAIRNESS_MESSAGES =
|
|
18
|
+
# fair-queue recv sockets. Symmetric with RoundRobin send batching.
|
|
19
|
+
FAIRNESS_MESSAGES = 256
|
|
20
20
|
|
|
21
21
|
|
|
22
22
|
# Max bytes read from one connection before yielding. Only counted
|
|
23
23
|
# for ZMTP connections (inproc skips the check). Complements
|
|
24
24
|
# {FAIRNESS_MESSAGES}: small-message floods are bounded by count,
|
|
25
|
-
# large-message floods by bytes.
|
|
26
|
-
FAIRNESS_BYTES =
|
|
25
|
+
# large-message floods by bytes. Symmetric with RoundRobin send batching.
|
|
26
|
+
FAIRNESS_BYTES = 512 * 1024
|
|
27
27
|
|
|
28
28
|
|
|
29
29
|
# Public entry point — callers use the class method.
|
|
30
30
|
#
|
|
31
31
|
# @param parent [Async::Task, Async::Barrier] parent to spawn under
|
|
32
32
|
# @param conn [Connection, Transport::Inproc::DirectPipe]
|
|
33
|
-
# @param recv_queue [
|
|
33
|
+
# @param recv_queue [Async::LimitedQueue]
|
|
34
34
|
# @param engine [Engine]
|
|
35
35
|
# @param transform [Proc, nil]
|
|
36
36
|
# @return [Async::Task, nil]
|
|
@@ -41,7 +41,7 @@ module OMQ
|
|
|
41
41
|
|
|
42
42
|
|
|
43
43
|
# @param conn [Connection, Transport::Inproc::DirectPipe]
|
|
44
|
-
# @param recv_queue [
|
|
44
|
+
# @param recv_queue [Async::LimitedQueue]
|
|
45
45
|
# @param engine [Engine]
|
|
46
46
|
#
|
|
47
47
|
def initialize(conn, recv_queue, engine)
|
|
@@ -104,7 +104,19 @@ module OMQ
|
|
|
104
104
|
recv_queue.enqueue(msg)
|
|
105
105
|
|
|
106
106
|
count += 1
|
|
107
|
-
|
|
107
|
+
|
|
108
|
+
# hot path
|
|
109
|
+
if count_bytes
|
|
110
|
+
if msg.size == 1
|
|
111
|
+
bytes += msg.first.bytesize
|
|
112
|
+
else
|
|
113
|
+
i, n = 0, msg.size
|
|
114
|
+
while i < n
|
|
115
|
+
bytes += msg[i].bytesize
|
|
116
|
+
i += 1
|
|
117
|
+
end
|
|
118
|
+
end
|
|
119
|
+
end
|
|
108
120
|
end
|
|
109
121
|
|
|
110
122
|
task.yield
|
|
@@ -132,13 +144,28 @@ module OMQ
|
|
|
132
144
|
loop do
|
|
133
145
|
count = 0
|
|
134
146
|
bytes = 0
|
|
147
|
+
|
|
135
148
|
while count < FAIRNESS_MESSAGES && bytes < FAIRNESS_BYTES
|
|
136
149
|
msg = conn.receive_message
|
|
137
150
|
engine.emit_verbose_msg_received(conn, msg)
|
|
138
151
|
recv_queue.enqueue(msg)
|
|
152
|
+
|
|
139
153
|
count += 1
|
|
140
|
-
|
|
154
|
+
|
|
155
|
+
# hot path
|
|
156
|
+
if count_bytes
|
|
157
|
+
if msg.size == 1
|
|
158
|
+
bytes += msg.first.bytesize
|
|
159
|
+
else
|
|
160
|
+
i, n = 0, msg.size
|
|
161
|
+
while i < n
|
|
162
|
+
bytes += msg[i].bytesize
|
|
163
|
+
i += 1
|
|
164
|
+
end
|
|
165
|
+
end
|
|
166
|
+
end
|
|
141
167
|
end
|
|
168
|
+
|
|
142
169
|
task.yield
|
|
143
170
|
end
|
|
144
171
|
rescue Async::Stop, Async::Cancel
|
data/lib/omq/engine.rb
CHANGED
|
@@ -273,7 +273,7 @@ module OMQ
|
|
|
273
273
|
# Starts a recv pump for a connection, or wires the inproc fast path.
|
|
274
274
|
#
|
|
275
275
|
# @param conn [Connection, Transport::Inproc::DirectPipe]
|
|
276
|
-
# @param recv_queue [
|
|
276
|
+
# @param recv_queue [Async::LimitedQueue]
|
|
277
277
|
# @yield [msg] optional per-message transform
|
|
278
278
|
# @return [Async::Task, nil]
|
|
279
279
|
#
|
|
@@ -587,6 +587,10 @@ module OMQ
|
|
|
587
587
|
break if deadline && (deadline - Async::Clock.now) <= 0
|
|
588
588
|
sleep 0.001
|
|
589
589
|
end
|
|
590
|
+
rescue Async::Stop
|
|
591
|
+
# Parent task is being cancelled — stop draining and let close
|
|
592
|
+
# proceed with the rest of teardown instead of propagating the
|
|
593
|
+
# cancellation out of the ensure path.
|
|
590
594
|
end
|
|
591
595
|
|
|
592
596
|
|
data/lib/omq/options.rb
CHANGED
|
@@ -10,9 +10,11 @@ module OMQ
|
|
|
10
10
|
DEFAULT_HWM = 1000
|
|
11
11
|
|
|
12
12
|
|
|
13
|
-
# @param linger [
|
|
13
|
+
# @param linger [Numeric] linger period in seconds on close
|
|
14
|
+
# (default Float::INFINITY = wait forever, matching libzmq).
|
|
15
|
+
# Pass 0 for immediate drop-on-close.
|
|
14
16
|
#
|
|
15
|
-
def initialize(linger:
|
|
17
|
+
def initialize(linger: Float::INFINITY)
|
|
16
18
|
@send_hwm = DEFAULT_HWM
|
|
17
19
|
@recv_hwm = DEFAULT_HWM
|
|
18
20
|
@linger = linger
|
|
@@ -39,7 +41,8 @@ module OMQ
|
|
|
39
41
|
# @!attribute recv_hwm
|
|
40
42
|
# @return [Integer] receive high water mark (default 1000, 0 = unbounded)
|
|
41
43
|
# @!attribute linger
|
|
42
|
-
# @return [
|
|
44
|
+
# @return [Numeric] linger period in seconds on close
|
|
45
|
+
# (Float::INFINITY = wait forever, 0 = immediate drop)
|
|
43
46
|
# @!attribute identity
|
|
44
47
|
# @return [String] socket identity for ROUTER addressing (default "")
|
|
45
48
|
# @!attribute router_mandatory
|
data/lib/omq/pair.rb
CHANGED
|
@@ -8,12 +8,25 @@ module OMQ
|
|
|
8
8
|
include Writable
|
|
9
9
|
|
|
10
10
|
# @param endpoints [String, nil] endpoint to bind/connect
|
|
11
|
-
# @param linger [
|
|
11
|
+
# @param linger [Numeric] linger period in seconds (Float::INFINITY = wait forever, 0 = drop)
|
|
12
|
+
# @param send_hwm [Integer, nil] send high water mark (nil uses default)
|
|
13
|
+
# @param recv_hwm [Integer, nil] receive high water mark (nil uses default)
|
|
14
|
+
# @param send_timeout [Numeric, nil] send timeout in seconds
|
|
15
|
+
# @param recv_timeout [Numeric, nil] receive timeout in seconds
|
|
12
16
|
# @param backend [Symbol, nil] :ruby (default) or :ffi
|
|
13
17
|
#
|
|
14
|
-
def initialize(endpoints = nil, linger:
|
|
15
|
-
|
|
18
|
+
def initialize(endpoints = nil, linger: Float::INFINITY,
|
|
19
|
+
send_hwm: nil, recv_hwm: nil,
|
|
20
|
+
send_timeout: nil, recv_timeout: nil,
|
|
21
|
+
backend: nil, &block)
|
|
22
|
+
init_engine(:PAIR, send_hwm: send_hwm, recv_hwm: recv_hwm,
|
|
23
|
+
send_timeout: send_timeout, recv_timeout: recv_timeout,
|
|
24
|
+
backend: backend)
|
|
25
|
+
@options.linger = linger
|
|
16
26
|
attach_endpoints(endpoints, default: :connect)
|
|
27
|
+
finalize_init(&block)
|
|
17
28
|
end
|
|
29
|
+
|
|
18
30
|
end
|
|
31
|
+
|
|
19
32
|
end
|
data/lib/omq/pub_sub.rb
CHANGED
|
@@ -7,15 +7,23 @@ module OMQ
|
|
|
7
7
|
include Writable
|
|
8
8
|
|
|
9
9
|
# @param endpoints [String, nil] endpoint to bind/connect
|
|
10
|
-
# @param linger [
|
|
10
|
+
# @param linger [Numeric] linger period in seconds (Float::INFINITY = wait forever, 0 = drop)
|
|
11
|
+
# @param send_hwm [Integer, nil] send high water mark
|
|
12
|
+
# @param send_timeout [Numeric, nil] send timeout in seconds
|
|
11
13
|
# @param on_mute [Symbol] mute strategy for slow subscribers
|
|
12
14
|
# @param conflate [Boolean] keep only latest message per topic
|
|
13
15
|
# @param backend [Symbol, nil] :ruby (default) or :ffi
|
|
14
16
|
#
|
|
15
|
-
def initialize(endpoints = nil, linger:
|
|
16
|
-
|
|
17
|
+
def initialize(endpoints = nil, linger: Float::INFINITY,
|
|
18
|
+
send_hwm: nil, send_timeout: nil,
|
|
19
|
+
on_mute: :drop_newest, conflate: false, backend: nil, &block)
|
|
20
|
+
init_engine(:PUB, send_hwm: send_hwm, send_timeout: send_timeout,
|
|
21
|
+
on_mute: on_mute, conflate: conflate, backend: backend)
|
|
22
|
+
@options.linger = linger
|
|
17
23
|
attach_endpoints(endpoints, default: :bind)
|
|
24
|
+
finalize_init(&block)
|
|
18
25
|
end
|
|
26
|
+
|
|
19
27
|
end
|
|
20
28
|
|
|
21
29
|
|
|
@@ -29,16 +37,21 @@ module OMQ
|
|
|
29
37
|
EVERYTHING = ''
|
|
30
38
|
|
|
31
39
|
|
|
32
|
-
# @param endpoints [String, nil]
|
|
33
|
-
# @param
|
|
40
|
+
# @param endpoints [String, nil] endpoint to bind/connect
|
|
41
|
+
# @param recv_hwm [Integer, nil] receive high water mark
|
|
42
|
+
# @param recv_timeout [Numeric, nil] receive timeout in seconds
|
|
34
43
|
# @param subscribe [String, nil] subscription prefix; +nil+ (default)
|
|
35
44
|
# means no subscription — call {#subscribe} explicitly.
|
|
36
45
|
# @param on_mute [Symbol] :block (default), :drop_newest, or :drop_oldest
|
|
46
|
+
# @param backend [Symbol, nil] :ruby (default) or :ffi
|
|
37
47
|
#
|
|
38
|
-
def initialize(endpoints = nil,
|
|
39
|
-
|
|
48
|
+
def initialize(endpoints = nil, recv_hwm: nil, recv_timeout: nil,
|
|
49
|
+
subscribe: nil, on_mute: :block, backend: nil, &block)
|
|
50
|
+
init_engine(:SUB, recv_hwm: recv_hwm, recv_timeout: recv_timeout,
|
|
51
|
+
on_mute: on_mute, backend: backend)
|
|
40
52
|
attach_endpoints(endpoints, default: :connect)
|
|
41
53
|
self.subscribe(subscribe) unless subscribe.nil?
|
|
54
|
+
finalize_init(&block)
|
|
42
55
|
end
|
|
43
56
|
|
|
44
57
|
|
|
@@ -60,6 +73,7 @@ module OMQ
|
|
|
60
73
|
def unsubscribe(prefix)
|
|
61
74
|
@engine.routing.unsubscribe(prefix)
|
|
62
75
|
end
|
|
76
|
+
|
|
63
77
|
end
|
|
64
78
|
|
|
65
79
|
|
|
@@ -70,14 +84,26 @@ module OMQ
|
|
|
70
84
|
include Writable
|
|
71
85
|
|
|
72
86
|
# @param endpoints [String, nil] endpoint to bind/connect
|
|
73
|
-
# @param linger [
|
|
87
|
+
# @param linger [Numeric] linger period in seconds (Float::INFINITY = wait forever, 0 = drop)
|
|
88
|
+
# @param send_hwm [Integer, nil] send high water mark
|
|
89
|
+
# @param recv_hwm [Integer, nil] receive high water mark
|
|
90
|
+
# @param send_timeout [Numeric, nil] send timeout in seconds
|
|
91
|
+
# @param recv_timeout [Numeric, nil] receive timeout in seconds
|
|
74
92
|
# @param on_mute [Symbol] mute strategy for slow subscribers
|
|
75
93
|
# @param backend [Symbol, nil] :ruby (default) or :ffi
|
|
76
94
|
#
|
|
77
|
-
def initialize(endpoints = nil, linger:
|
|
78
|
-
|
|
95
|
+
def initialize(endpoints = nil, linger: Float::INFINITY,
|
|
96
|
+
send_hwm: nil, recv_hwm: nil,
|
|
97
|
+
send_timeout: nil, recv_timeout: nil,
|
|
98
|
+
on_mute: :drop_newest, backend: nil, &block)
|
|
99
|
+
init_engine(:XPUB, send_hwm: send_hwm, recv_hwm: recv_hwm,
|
|
100
|
+
send_timeout: send_timeout, recv_timeout: recv_timeout,
|
|
101
|
+
on_mute: on_mute, backend: backend)
|
|
102
|
+
@options.linger = linger
|
|
79
103
|
attach_endpoints(endpoints, default: :bind)
|
|
104
|
+
finalize_init(&block)
|
|
80
105
|
end
|
|
106
|
+
|
|
81
107
|
end
|
|
82
108
|
|
|
83
109
|
|
|
@@ -87,17 +113,30 @@ module OMQ
|
|
|
87
113
|
include Readable
|
|
88
114
|
include Writable
|
|
89
115
|
|
|
90
|
-
# @param endpoints [String, nil]
|
|
91
|
-
# @param linger [
|
|
116
|
+
# @param endpoints [String, nil] endpoint to bind/connect
|
|
117
|
+
# @param linger [Numeric] linger period in seconds (Float::INFINITY = wait forever, 0 = drop)
|
|
118
|
+
# @param send_hwm [Integer, nil] send high water mark
|
|
119
|
+
# @param recv_hwm [Integer, nil] receive high water mark
|
|
120
|
+
# @param send_timeout [Numeric, nil] send timeout in seconds
|
|
121
|
+
# @param recv_timeout [Numeric, nil] receive timeout in seconds
|
|
92
122
|
# @param subscribe [String, nil] subscription prefix; +nil+ (default)
|
|
93
123
|
# means no subscription — send a subscribe frame explicitly.
|
|
94
124
|
# @param on_mute [Symbol] mute strategy (:block, :drop_newest, :drop_oldest)
|
|
95
125
|
# @param backend [Symbol, nil] :ruby (default) or :ffi
|
|
96
126
|
#
|
|
97
|
-
def initialize(endpoints = nil, linger:
|
|
98
|
-
|
|
127
|
+
def initialize(endpoints = nil, linger: Float::INFINITY,
|
|
128
|
+
send_hwm: nil, recv_hwm: nil,
|
|
129
|
+
send_timeout: nil, recv_timeout: nil,
|
|
130
|
+
subscribe: nil, on_mute: :block, backend: nil, &block)
|
|
131
|
+
init_engine(:XSUB, send_hwm: send_hwm, recv_hwm: recv_hwm,
|
|
132
|
+
send_timeout: send_timeout, recv_timeout: recv_timeout,
|
|
133
|
+
on_mute: on_mute, backend: backend)
|
|
134
|
+
@options.linger = linger
|
|
99
135
|
attach_endpoints(endpoints, default: :connect)
|
|
100
136
|
send("\x01#{subscribe}".b) unless subscribe.nil?
|
|
137
|
+
finalize_init(&block)
|
|
101
138
|
end
|
|
139
|
+
|
|
102
140
|
end
|
|
141
|
+
|
|
103
142
|
end
|
data/lib/omq/push_pull.rb
CHANGED
|
@@ -7,15 +7,18 @@ module OMQ
|
|
|
7
7
|
include Writable
|
|
8
8
|
|
|
9
9
|
# @param endpoints [String, nil] endpoint to bind/connect
|
|
10
|
-
# @param linger [
|
|
10
|
+
# @param linger [Numeric] linger period in seconds (Float::INFINITY = wait forever, 0 = drop)
|
|
11
11
|
# @param send_hwm [Integer, nil] send high water mark (nil uses default)
|
|
12
12
|
# @param send_timeout [Numeric, nil] send timeout in seconds
|
|
13
13
|
# @param backend [Symbol, nil] :ruby (default) or :ffi
|
|
14
14
|
#
|
|
15
|
-
def initialize(endpoints = nil, linger:
|
|
16
|
-
init_engine(:PUSH,
|
|
15
|
+
def initialize(endpoints = nil, linger: Float::INFINITY, send_hwm: nil, send_timeout: nil, backend: nil, &block)
|
|
16
|
+
init_engine(:PUSH, send_hwm: send_hwm, send_timeout: send_timeout, backend: backend)
|
|
17
|
+
@options.linger = linger
|
|
17
18
|
attach_endpoints(endpoints, default: :connect)
|
|
19
|
+
finalize_init(&block)
|
|
18
20
|
end
|
|
21
|
+
|
|
19
22
|
end
|
|
20
23
|
|
|
21
24
|
|
|
@@ -25,14 +28,16 @@ module OMQ
|
|
|
25
28
|
include Readable
|
|
26
29
|
|
|
27
30
|
# @param endpoints [String, nil] endpoint to bind/connect
|
|
28
|
-
# @param linger [Integer] linger period in seconds
|
|
29
31
|
# @param recv_hwm [Integer, nil] receive high water mark (nil uses default)
|
|
30
32
|
# @param recv_timeout [Numeric, nil] receive timeout in seconds
|
|
31
33
|
# @param backend [Symbol, nil] :ruby (default) or :ffi
|
|
32
34
|
#
|
|
33
|
-
def initialize(endpoints = nil,
|
|
34
|
-
init_engine(:PULL,
|
|
35
|
+
def initialize(endpoints = nil, recv_hwm: nil, recv_timeout: nil, backend: nil, &block)
|
|
36
|
+
init_engine(:PULL, recv_hwm: recv_hwm, recv_timeout: recv_timeout, backend: backend)
|
|
35
37
|
attach_endpoints(endpoints, default: :bind)
|
|
38
|
+
finalize_init(&block)
|
|
36
39
|
end
|
|
40
|
+
|
|
37
41
|
end
|
|
42
|
+
|
|
38
43
|
end
|