omq 0.27.0 → 0.28.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/CHANGELOG.md +41 -2
- data/README.md +64 -42
- data/lib/omq/backend.rb +23 -0
- data/lib/omq/engine.rb +6 -6
- data/lib/omq/pair.rb +1 -1
- data/lib/omq/pub_sub.rb +4 -4
- data/lib/omq/push_pull.rb +2 -2
- data/lib/omq/req_rep.rb +2 -2
- data/lib/omq/router_dealer.rb +2 -2
- data/lib/omq/routing/fan_out.rb +40 -11
- data/lib/omq/socket.rb +19 -9
- data/lib/omq/transport/inproc.rb +9 -7
- data/lib/omq/version.rb +1 -1
- data/lib/omq.rb +2 -0
- metadata +6 -8
- data/lib/omq/ffi/engine.rb +0 -646
- data/lib/omq/ffi/libzmq.rb +0 -134
- data/lib/omq/ffi.rb +0 -12
checksums.yaml
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
SHA256:
|
|
3
|
-
metadata.gz:
|
|
4
|
-
data.tar.gz:
|
|
3
|
+
metadata.gz: ec6bb2de8ced07ba0d20f272256c2d163bb2e192f01c292b6d8431c792c4a289
|
|
4
|
+
data.tar.gz: 662ee0364dd2e6ac7ba28822cfc08d5746bcd3cdf862c41ed48a33472b254bce
|
|
5
5
|
SHA512:
|
|
6
|
-
metadata.gz:
|
|
7
|
-
data.tar.gz:
|
|
6
|
+
metadata.gz: 82b84f4c8e9e4b54d7ec7a4f9d9beca5dc0666a9491220e84c47a8956f6c275ea2172d9dfb509e0f7dcfd16d6a9429dd902703ffbacfc7c28a5b03939818059a
|
|
7
|
+
data.tar.gz: 26628cfdddb28d49bfbc47e4d7bb8cb9878f0e283f64c8669dd622db713f3fdbceac4776e184fd87cd6c691d92ebc729ead807bcf5e6a3a8e9338a31496e05d9
|
data/CHANGELOG.md
CHANGED
|
@@ -1,5 +1,44 @@
|
|
|
1
1
|
# Changelog
|
|
2
2
|
|
|
3
|
+
## [Unreleased]
|
|
4
|
+
|
|
5
|
+
## 0.28.1 - 2026-07-23
|
|
6
|
+
|
|
7
|
+
### Changed
|
|
8
|
+
|
|
9
|
+
- Consolidated optional OMQ gems into this repository under `gems/`.
|
|
10
|
+
- Consolidated `protocol-zmtp` into this repository under
|
|
11
|
+
`gems/protocol-zmtp`.
|
|
12
|
+
- Moved the libzmq backend out of the `omq` gem and into
|
|
13
|
+
`omq-backend-libzmq`. The old `require "omq/ffi"` and `backend: :ffi`
|
|
14
|
+
names still work when the backend gem is installed.
|
|
15
|
+
- Release CI now publishes packages from prefixed tags such as
|
|
16
|
+
`omq-v0.28.0`, `omq-lz4-v0.3.1`, and
|
|
17
|
+
`omq-backend-rust-v0.1.0`.
|
|
18
|
+
- Removed `omq-blake3zmq` from the development dependency set.
|
|
19
|
+
- Bumped the `protocol-zmtp` dependency requirement to `~> 0.10`.
|
|
20
|
+
|
|
21
|
+
## 0.28.0 — 2026-07-04
|
|
22
|
+
|
|
23
|
+
### Changed
|
|
24
|
+
|
|
25
|
+
- **Ruby in-process transport renamed from `inproc://` to `ruby://`.**
|
|
26
|
+
The `inproc://` scheme is now reserved for native backends (FFI/libzmq,
|
|
27
|
+
Rust/omq-tokio) which maintain their own in-process registries.
|
|
28
|
+
All existing `inproc://` endpoints must be updated to `ruby://`.
|
|
29
|
+
|
|
30
|
+
### Added
|
|
31
|
+
|
|
32
|
+
- **`:rust` backend branch in `Socket#init_engine`.** Sockets can now be
|
|
33
|
+
created with `backend: :rust` to use the Rust/omq-tokio engine
|
|
34
|
+
(requires `omq-rust` and the native extension).
|
|
35
|
+
- **Cross-backend benchmarks and chart generation.** New bench scripts
|
|
36
|
+
(`bench_pushpull.rb`, `bench_reqrep.rb`, `bench_pubsub.rb`,
|
|
37
|
+
`bench_peer.rb`) compare pure Ruby, FFI, and Rust backends.
|
|
38
|
+
`bench/chart_helper.rb` generates SVG comparison charts.
|
|
39
|
+
- **Soak test.** `test/omq/soak_test.rb` exercises sustained throughput
|
|
40
|
+
across backends and patterns.
|
|
41
|
+
|
|
3
42
|
## 0.27.0 — 2026-04-20
|
|
4
43
|
|
|
5
44
|
### Added
|
|
@@ -1320,7 +1359,7 @@ Combined effect of caller-owns-data + Reactor fast path on inproc:
|
|
|
1320
1359
|
- **Protocol layer extracted into protocol-zmtp gem** — Codec (Frame,
|
|
1321
1360
|
Greeting, Command), Connection, Mechanism::Null, Mechanism::Curve,
|
|
1322
1361
|
ValidPeers, and Z85 now live in the
|
|
1323
|
-
[protocol-zmtp](
|
|
1362
|
+
[protocol-zmtp](gems/protocol-zmtp) gem. OMQ
|
|
1324
1363
|
re-exports them under `OMQ::ZMTP::` for backwards compatibility.
|
|
1325
1364
|
protocol-zmtp has zero runtime dependencies.
|
|
1326
1365
|
- **Unified CURVE mechanism** — one implementation with a pluggable
|
|
@@ -1834,4 +1873,4 @@ Initial release. Pure Ruby implementation of ZMTP 3.1 (ZeroMQ) using Async.
|
|
|
1834
1873
|
- Linger on close (drain send queue before closing)
|
|
1835
1874
|
- `max_message_size` enforcement
|
|
1836
1875
|
- Works inside Async reactors or standalone (shared IO thread)
|
|
1837
|
-
- Optional CURVE encryption via the [protocol-zmtp](
|
|
1876
|
+
- Optional CURVE encryption via the [protocol-zmtp](gems/protocol-zmtp) gem
|
data/README.md
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
|
-
# ØMQ
|
|
1
|
+
# ØMQ.rb
|
|
2
2
|
|
|
3
|
-
[](https://github.com/zeromq/omq/actions/workflows/ci.yml)
|
|
3
|
+
[](https://github.com/zeromq/omq.rb/actions/workflows/ci.yml)
|
|
4
4
|
[](https://rubygems.org/gems/omq)
|
|
5
5
|
[](LICENSE)
|
|
6
6
|
[](https://www.ruby-lang.org)
|
|
@@ -9,47 +9,48 @@
|
|
|
9
9
|
>
|
|
10
10
|
> **8.7 µs** inproc latency | **51 µs** ipc | **64 µs** tcp
|
|
11
11
|
>
|
|
12
|
-
> Ruby 4.0 + YJIT on a Linux VM
|
|
12
|
+
> Ruby 4.0 + YJIT on a Linux VM. See [`bench/`](bench/) for full results
|
|
13
13
|
|
|
14
|
-
`gem install omq` and you're done. No libzmq, no compiler, no system packages
|
|
14
|
+
`gem install omq` and you're done. No libzmq, no compiler, no system packages,
|
|
15
15
|
just Ruby talking to every other ZeroMQ peer out there.
|
|
16
16
|
|
|
17
|
-
ØMQ gives your Ruby processes a way to talk to each other
|
|
18
|
-
else speaking ZeroMQ
|
|
17
|
+
ØMQ gives your Ruby processes a way to talk to each other, and to anything
|
|
18
|
+
else speaking ZeroMQ, without a broker in the middle. Same API whether they
|
|
19
19
|
live in the same process, on the same machine, or across the network.
|
|
20
|
-
Reconnects, queuing, and back-pressure are handled for you
|
|
20
|
+
Reconnects, queuing, and back-pressure are handled for you. You write the
|
|
21
21
|
interesting part.
|
|
22
22
|
|
|
23
|
-
New to ZeroMQ? Start with [GETTING_STARTED.md](doc/GETTING_STARTED.md)
|
|
23
|
+
New to ZeroMQ? Start with [GETTING_STARTED.md](doc/GETTING_STARTED.md), a ~30 min
|
|
24
24
|
walkthrough of every major pattern with working code.
|
|
25
25
|
|
|
26
26
|
## Highlights
|
|
27
27
|
|
|
28
|
-
- **Zero dependencies on C
|
|
28
|
+
- **Zero dependencies on C**: no extensions, no FFI, no libzmq. `gem install`
|
|
29
29
|
just works everywhere
|
|
30
|
-
- **Fast
|
|
30
|
+
- **Fast**: YJIT-optimized hot paths, batched sends, GC-tuned allocations,
|
|
31
31
|
buffered I/O via [io-stream](https://github.com/socketry/io-stream),
|
|
32
32
|
direct-pipe inproc bypass
|
|
33
|
-
- **[`omq` CLI](https://github.com/paddor/omq-cli)
|
|
33
|
+
- **[`omq` CLI](https://github.com/paddor/omq-cli)**: a powerful swiss army
|
|
34
34
|
knife for ØMQ. `gem install omq-cli`
|
|
35
|
-
- **Every socket pattern
|
|
35
|
+
- **Every socket pattern**: req/rep, pub/sub, push/pull, dealer/router,
|
|
36
36
|
xpub/xsub, pair, and all draft types
|
|
37
|
-
- **Every transport
|
|
37
|
+
- **Every transport**: tcp, ipc (Unix domain sockets), inproc (in-process
|
|
38
38
|
queues)
|
|
39
|
-
- **Async-native
|
|
40
|
-
- **Works outside Async too
|
|
39
|
+
- **Async-native**: built on fibers, non-blocking from the ground up
|
|
40
|
+
- **Works outside Async too**: a shared IO thread handles sockets for callers
|
|
41
41
|
that aren't inside a reactor, so simple scripts just work
|
|
42
|
-
- **Wire-compatible
|
|
42
|
+
- **Wire-compatible**: interoperates with libzmq, pyzmq, CZMQ, zmq.rs over tcp
|
|
43
43
|
and ipc
|
|
44
|
-
- **Bind/connect order doesn't matter
|
|
44
|
+
- **Bind/connect order doesn't matter**: connect before bind, bind before
|
|
45
45
|
connect, peers come and go. ZeroMQ reconnects automatically and queued
|
|
46
46
|
messages drain when peers arrive
|
|
47
47
|
|
|
48
48
|
For architecture internals, see [DESIGN.md](doc/DESIGN.md).
|
|
49
|
+
If you're interested in the steps taken to reach the current performance, read [PERFORMANCE.md](doc/PERFORMANCE.md).
|
|
49
50
|
|
|
50
51
|
## Install
|
|
51
52
|
|
|
52
|
-
No system libraries needed
|
|
53
|
+
No system libraries needed. Just Ruby:
|
|
53
54
|
|
|
54
55
|
```sh
|
|
55
56
|
gem install omq
|
|
@@ -115,7 +116,7 @@ end
|
|
|
115
116
|
|
|
116
117
|
### Without Async (IO thread)
|
|
117
118
|
|
|
118
|
-
OMQ spawns a shared `omq-io` thread when used outside an Async reactor
|
|
119
|
+
OMQ spawns a shared `omq-io` thread when used outside an Async reactor. No
|
|
119
120
|
boilerplate needed:
|
|
120
121
|
|
|
121
122
|
```ruby
|
|
@@ -143,7 +144,7 @@ All sockets expose an `Async::Queue`-inspired interface:
|
|
|
143
144
|
| `enqueue(item)` / `push(item)` | `enqueue(msg)` / `push(msg)` | Also: `send(msg)`, `<< msg` |
|
|
144
145
|
| `dequeue(timeout:)` / `pop(timeout:)` | `dequeue(timeout:)` / `pop(timeout:)` | Defaults to socket's `read_timeout` |
|
|
145
146
|
| `wait` | `wait` | Blocks indefinitely (ignores `read_timeout`) |
|
|
146
|
-
| `each` | `each` | Yields messages
|
|
147
|
+
| `each` | `each` | Yields messages, returns on close or timeout |
|
|
147
148
|
|
|
148
149
|
```ruby
|
|
149
150
|
pull = OMQ::PULL.bind('inproc://work')
|
|
@@ -157,13 +158,13 @@ end
|
|
|
157
158
|
## Socket Types
|
|
158
159
|
|
|
159
160
|
All sockets are thread-safe. Default HWM is 1000 messages per socket.
|
|
160
|
-
`max_message_size` defaults to **`nil` (unlimited)
|
|
161
|
-
`socket.max_message_size = N` to cap inbound frames at `N` bytes
|
|
161
|
+
`max_message_size` defaults to **`nil` (unlimited)**. Set
|
|
162
|
+
`socket.max_message_size = N` to cap inbound frames at `N` bytes. Oversized
|
|
162
163
|
frames cause the connection to be dropped before the body is read from the
|
|
163
164
|
wire. Classes live under `OMQ::` (alias: `ØMQ`).
|
|
164
165
|
|
|
165
166
|
**Received messages are frozen** across all transports (inproc, ipc, tcp).
|
|
166
|
-
The array returned by `#receive` and every part inside it is frozen
|
|
167
|
+
The array returned by `#receive` and every part inside it is frozen.
|
|
167
168
|
mutating a received part raises `FrozenError` rather than silently
|
|
168
169
|
corrupting a shared reference on the inproc fast path. `dup` a part if
|
|
169
170
|
you need to mutate it.
|
|
@@ -187,7 +188,7 @@ you need to mutate it.
|
|
|
187
188
|
|
|
188
189
|
#### Draft (single-frame only)
|
|
189
190
|
|
|
190
|
-
Bundled with `omq` but not loaded by `require "omq"
|
|
191
|
+
Bundled with `omq` but not loaded by `require "omq"`. Opt in with the matching
|
|
191
192
|
`require` line. See [`doc/socket-types/`](doc/socket-types/) for per-pattern
|
|
192
193
|
usage.
|
|
193
194
|
|
|
@@ -215,32 +216,54 @@ See the [omq-cli README](https://github.com/paddor/omq-cli) for full documentati
|
|
|
215
216
|
|
|
216
217
|
## Optional libzmq backend
|
|
217
218
|
|
|
218
|
-
|
|
219
|
-
backed by libzmq instead of the pure Ruby ZMTP stack. Useful
|
|
220
|
-
need libzmq-specific features or for verifying wire
|
|
219
|
+
Install `omq-backend-libzmq` for a Ruby 4.0+ libzmq backend. Same socket
|
|
220
|
+
API, but backed by libzmq instead of the pure Ruby ZMTP stack. Useful
|
|
221
|
+
when you need libzmq-specific features or for verifying wire
|
|
222
|
+
compatibility.
|
|
221
223
|
|
|
222
224
|
```ruby
|
|
223
|
-
require "omq/
|
|
224
|
-
push = OMQ::PUSH.new(backend: :
|
|
225
|
+
require "omq/backend/libzmq"
|
|
226
|
+
push = OMQ::PUSH.new(backend: :libzmq)
|
|
225
227
|
```
|
|
226
228
|
|
|
227
229
|
Requires the `ffi` gem and a system libzmq 4.x. `ffi` is not a runtime
|
|
228
|
-
dependency of `omq
|
|
229
|
-
|
|
230
|
+
dependency of `omq`. The old `require "omq/ffi"` and `backend: :ffi`
|
|
231
|
+
names still work.
|
|
230
232
|
|
|
231
233
|
## Companion Gems
|
|
232
234
|
|
|
233
|
-
- **[
|
|
235
|
+
- **[protocol-zmtp](gems/protocol-zmtp)**: ZMTP 3.1 frame codec,
|
|
236
|
+
greeting, security mechanisms, and connection handshake. It lives in
|
|
237
|
+
this monorepo because `omq` uses it as its wire-protocol module. Most
|
|
238
|
+
users never require it directly.
|
|
239
|
+
- **[omq-ractor](gems/omq-ractor)**: bridge OMQ sockets
|
|
234
240
|
into Ruby Ractors for true parallel processing across cores. I/O stays on the
|
|
235
241
|
main Ractor, worker Ractors do pure computation.
|
|
236
242
|
|
|
243
|
+
### Compressed transports
|
|
244
|
+
|
|
245
|
+
Drop-in TCP replacements that compress every message part on the wire. Both
|
|
246
|
+
peers must use the same scheme. A `tcp://` peer cannot talk to a
|
|
247
|
+
compressed peer. The ZMTP handshake itself runs over plain TCP. Only
|
|
248
|
+
post-handshake message parts are compressed.
|
|
249
|
+
|
|
250
|
+
- **[omq-lz4](gems/omq-lz4)**: `lz4+tcp://`.
|
|
251
|
+
LZ4 per-part compression with user dictionaries and `auto_dict:`
|
|
252
|
+
training from early traffic. Use for new compressed TCP work.
|
|
253
|
+
- **[omq-zstd](gems/omq-zstd)**: experimental `zstd+tcp://`.
|
|
254
|
+
Zstandard per-part compression kept for research and comparison. Prefer
|
|
255
|
+
`omq-lz4` for new compressed TCP work. OMQ.rs removed `zstd+tcp://`
|
|
256
|
+
to reduce transport complexity after lz4rip gained dictionary training.
|
|
257
|
+
|
|
237
258
|
### Protocol extensions (RFCs)
|
|
238
259
|
|
|
239
|
-
Optional plug-ins that extend the ZMTP wire protocol. Each is a separate gem
|
|
260
|
+
Optional plug-ins that extend the ZMTP wire protocol. Each is a separate gem.
|
|
240
261
|
load the ones you need.
|
|
241
262
|
|
|
242
|
-
- **[omq-
|
|
243
|
-
|
|
263
|
+
- **[omq-qos](gems/omq-qos)**: experimental MQTT-style at-least-once
|
|
264
|
+
delivery. Receivers send an ACK command frame keyed by xxHash digest.
|
|
265
|
+
unacked messages re-enqueue to the next peer on disconnect. PUSH/PULL,
|
|
266
|
+
SCATTER/GATHER, REQ/REP only. Fan-out patterns are out of scope.
|
|
244
267
|
|
|
245
268
|
## Development
|
|
246
269
|
|
|
@@ -252,17 +275,16 @@ bundle exec rake
|
|
|
252
275
|
### Full development setup
|
|
253
276
|
|
|
254
277
|
Set `OMQ_DEV=1` to tell Bundler to load sibling projects from source
|
|
255
|
-
(
|
|
256
|
-
This is required for running
|
|
257
|
-
the stack.
|
|
278
|
+
(nuckle, lz4rip, zrip, etc.) instead of released gems. `protocol-zmtp`
|
|
279
|
+
already lives in `gems/protocol-zmtp`. This is required for running
|
|
280
|
+
benchmarks and for testing changes across the stack.
|
|
258
281
|
|
|
259
282
|
```sh
|
|
260
283
|
# clone OMQ and its sibling repos into the same parent directory
|
|
261
|
-
git clone https://github.com/
|
|
262
|
-
git clone https://github.com/paddor/protocol-zmtp.git
|
|
263
|
-
git clone https://github.com/paddor/omq-zstd.git
|
|
264
|
-
git clone https://github.com/paddor/omq-ractor.git
|
|
284
|
+
git clone https://github.com/zeromq/omq.rb.git omq
|
|
265
285
|
git clone https://github.com/paddor/nuckle.git
|
|
286
|
+
git clone https://github.com/paddor/lz4rip-rb.git lz4rip
|
|
287
|
+
git clone https://github.com/paddor/zrip-rb.git zrip
|
|
266
288
|
|
|
267
289
|
cd omq
|
|
268
290
|
OMQ_DEV=1 bundle install
|
data/lib/omq/backend.rb
ADDED
|
@@ -0,0 +1,23 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
module OMQ
|
|
4
|
+
module Backend
|
|
5
|
+
@engines = {}
|
|
6
|
+
|
|
7
|
+
class << self
|
|
8
|
+
def register(name, engine_class)
|
|
9
|
+
@engines[name.to_sym] = engine_class
|
|
10
|
+
end
|
|
11
|
+
|
|
12
|
+
|
|
13
|
+
def fetch(name)
|
|
14
|
+
@engines[name.to_sym]
|
|
15
|
+
end
|
|
16
|
+
|
|
17
|
+
|
|
18
|
+
def registered?(name)
|
|
19
|
+
@engines.key?(name.to_sym)
|
|
20
|
+
end
|
|
21
|
+
end
|
|
22
|
+
end
|
|
23
|
+
end
|
data/lib/omq/engine.rb
CHANGED
|
@@ -166,7 +166,7 @@ module OMQ
|
|
|
166
166
|
end
|
|
167
167
|
|
|
168
168
|
|
|
169
|
-
# Spawns
|
|
169
|
+
# Spawns a ruby:// reconnect retry task under the socket's parent task.
|
|
170
170
|
#
|
|
171
171
|
# @param endpoint [String]
|
|
172
172
|
# @yield [interval] the retry loop body
|
|
@@ -185,7 +185,7 @@ module OMQ
|
|
|
185
185
|
|
|
186
186
|
# Binds to an endpoint.
|
|
187
187
|
#
|
|
188
|
-
# @param endpoint [String] e.g. "tcp://127.0.0.1:5555", "
|
|
188
|
+
# @param endpoint [String] e.g. "tcp://127.0.0.1:5555", "ruby://foo"
|
|
189
189
|
# @return [URI::Generic] resolved endpoint URI (with auto-selected port for "tcp://host:0")
|
|
190
190
|
# @raise [ArgumentError] on unsupported transport
|
|
191
191
|
#
|
|
@@ -216,11 +216,11 @@ module OMQ
|
|
|
216
216
|
capture_parent_task(parent: parent)
|
|
217
217
|
validate_endpoint!(endpoint)
|
|
218
218
|
|
|
219
|
-
if endpoint.start_with?("
|
|
220
|
-
#
|
|
219
|
+
if endpoint.start_with?("ruby://")
|
|
220
|
+
# Ruby in-process connect is synchronous and instant — no Dialer
|
|
221
221
|
transport = transport_for(endpoint)
|
|
222
222
|
transport.connect(endpoint, self, **opts)
|
|
223
|
-
@dialers[endpoint] = :
|
|
223
|
+
@dialers[endpoint] = :ruby # sentinel for reconnect intent
|
|
224
224
|
else
|
|
225
225
|
transport = transport_for(endpoint)
|
|
226
226
|
@dialers[endpoint] = transport.dialer(endpoint, self, **opts)
|
|
@@ -603,7 +603,7 @@ module OMQ
|
|
|
603
603
|
|
|
604
604
|
# Looks up the transport module for an endpoint URI.
|
|
605
605
|
#
|
|
606
|
-
# @param endpoint [String] endpoint URI (e.g. "tcp://...", "
|
|
606
|
+
# @param endpoint [String] endpoint URI (e.g. "tcp://...", "ruby://...")
|
|
607
607
|
# @return [Module] the transport module
|
|
608
608
|
# @raise [ArgumentError] if the scheme is not registered
|
|
609
609
|
#
|
data/lib/omq/pair.rb
CHANGED
|
@@ -13,7 +13,7 @@ module OMQ
|
|
|
13
13
|
# @param recv_hwm [Integer, nil] receive high water mark (nil uses default)
|
|
14
14
|
# @param send_timeout [Numeric, nil] send timeout in seconds
|
|
15
15
|
# @param recv_timeout [Numeric, nil] receive timeout in seconds
|
|
16
|
-
# @param backend [Symbol, nil] :ruby
|
|
16
|
+
# @param backend [Symbol, nil] registered backend name (:ruby by default)
|
|
17
17
|
#
|
|
18
18
|
def initialize(endpoints = nil, linger: Float::INFINITY,
|
|
19
19
|
send_hwm: nil, recv_hwm: nil,
|
data/lib/omq/pub_sub.rb
CHANGED
|
@@ -12,7 +12,7 @@ module OMQ
|
|
|
12
12
|
# @param send_timeout [Numeric, nil] send timeout in seconds
|
|
13
13
|
# @param on_mute [Symbol] mute strategy for slow subscribers
|
|
14
14
|
# @param conflate [Boolean] keep only latest message per topic
|
|
15
|
-
# @param backend [Symbol, nil] :ruby
|
|
15
|
+
# @param backend [Symbol, nil] registered backend name (:ruby by default)
|
|
16
16
|
#
|
|
17
17
|
def initialize(endpoints = nil, linger: Float::INFINITY,
|
|
18
18
|
send_hwm: nil, send_timeout: nil,
|
|
@@ -43,7 +43,7 @@ module OMQ
|
|
|
43
43
|
# @param subscribe [String, nil] subscription prefix; +nil+ (default)
|
|
44
44
|
# means no subscription — call {#subscribe} explicitly.
|
|
45
45
|
# @param on_mute [Symbol] :block (default), :drop_newest, or :drop_oldest
|
|
46
|
-
# @param backend [Symbol, nil] :ruby
|
|
46
|
+
# @param backend [Symbol, nil] registered backend name (:ruby by default)
|
|
47
47
|
#
|
|
48
48
|
def initialize(endpoints = nil, recv_hwm: nil, recv_timeout: nil,
|
|
49
49
|
subscribe: nil, on_mute: :block, backend: nil, &block)
|
|
@@ -90,7 +90,7 @@ module OMQ
|
|
|
90
90
|
# @param send_timeout [Numeric, nil] send timeout in seconds
|
|
91
91
|
# @param recv_timeout [Numeric, nil] receive timeout in seconds
|
|
92
92
|
# @param on_mute [Symbol] mute strategy for slow subscribers
|
|
93
|
-
# @param backend [Symbol, nil] :ruby
|
|
93
|
+
# @param backend [Symbol, nil] registered backend name (:ruby by default)
|
|
94
94
|
#
|
|
95
95
|
def initialize(endpoints = nil, linger: Float::INFINITY,
|
|
96
96
|
send_hwm: nil, recv_hwm: nil,
|
|
@@ -122,7 +122,7 @@ module OMQ
|
|
|
122
122
|
# @param subscribe [String, nil] subscription prefix; +nil+ (default)
|
|
123
123
|
# means no subscription — send a subscribe frame explicitly.
|
|
124
124
|
# @param on_mute [Symbol] mute strategy (:block, :drop_newest, :drop_oldest)
|
|
125
|
-
# @param backend [Symbol, nil] :ruby
|
|
125
|
+
# @param backend [Symbol, nil] registered backend name (:ruby by default)
|
|
126
126
|
#
|
|
127
127
|
def initialize(endpoints = nil, linger: Float::INFINITY,
|
|
128
128
|
send_hwm: nil, recv_hwm: nil,
|
data/lib/omq/push_pull.rb
CHANGED
|
@@ -10,7 +10,7 @@ module OMQ
|
|
|
10
10
|
# @param linger [Numeric] linger period in seconds (Float::INFINITY = wait forever, 0 = drop)
|
|
11
11
|
# @param send_hwm [Integer, nil] send high water mark (nil uses default)
|
|
12
12
|
# @param send_timeout [Numeric, nil] send timeout in seconds
|
|
13
|
-
# @param backend [Symbol, nil] :ruby
|
|
13
|
+
# @param backend [Symbol, nil] registered backend name (:ruby by default)
|
|
14
14
|
#
|
|
15
15
|
def initialize(endpoints = nil, linger: Float::INFINITY, send_hwm: nil, send_timeout: nil, backend: nil, &block)
|
|
16
16
|
init_engine(:PUSH, send_hwm: send_hwm, send_timeout: send_timeout, backend: backend)
|
|
@@ -30,7 +30,7 @@ module OMQ
|
|
|
30
30
|
# @param endpoints [String, nil] endpoint to bind/connect
|
|
31
31
|
# @param recv_hwm [Integer, nil] receive high water mark (nil uses default)
|
|
32
32
|
# @param recv_timeout [Numeric, nil] receive timeout in seconds
|
|
33
|
-
# @param backend [Symbol, nil] :ruby
|
|
33
|
+
# @param backend [Symbol, nil] registered backend name (:ruby by default)
|
|
34
34
|
#
|
|
35
35
|
def initialize(endpoints = nil, recv_hwm: nil, recv_timeout: nil, backend: nil, &block)
|
|
36
36
|
init_engine(:PULL, recv_hwm: recv_hwm, recv_timeout: recv_timeout, backend: backend)
|
data/lib/omq/req_rep.rb
CHANGED
|
@@ -13,7 +13,7 @@ module OMQ
|
|
|
13
13
|
# @param recv_hwm [Integer, nil] receive high water mark
|
|
14
14
|
# @param send_timeout [Numeric, nil] send timeout in seconds
|
|
15
15
|
# @param recv_timeout [Numeric, nil] receive timeout in seconds
|
|
16
|
-
# @param backend [Symbol, nil] :ruby
|
|
16
|
+
# @param backend [Symbol, nil] registered backend name (:ruby by default)
|
|
17
17
|
#
|
|
18
18
|
def initialize(endpoints = nil, linger: Float::INFINITY,
|
|
19
19
|
send_hwm: nil, recv_hwm: nil,
|
|
@@ -42,7 +42,7 @@ module OMQ
|
|
|
42
42
|
# @param recv_hwm [Integer, nil] receive high water mark
|
|
43
43
|
# @param send_timeout [Numeric, nil] send timeout in seconds
|
|
44
44
|
# @param recv_timeout [Numeric, nil] receive timeout in seconds
|
|
45
|
-
# @param backend [Symbol, nil] :ruby
|
|
45
|
+
# @param backend [Symbol, nil] registered backend name (:ruby by default)
|
|
46
46
|
#
|
|
47
47
|
def initialize(endpoints = nil, linger: Float::INFINITY,
|
|
48
48
|
send_hwm: nil, recv_hwm: nil,
|
data/lib/omq/router_dealer.rb
CHANGED
|
@@ -13,7 +13,7 @@ module OMQ
|
|
|
13
13
|
# @param recv_hwm [Integer, nil] receive high water mark
|
|
14
14
|
# @param send_timeout [Numeric, nil] send timeout in seconds
|
|
15
15
|
# @param recv_timeout [Numeric, nil] receive timeout in seconds
|
|
16
|
-
# @param backend [Symbol, nil] :ruby
|
|
16
|
+
# @param backend [Symbol, nil] registered backend name (:ruby by default)
|
|
17
17
|
#
|
|
18
18
|
def initialize(endpoints = nil, linger: Float::INFINITY,
|
|
19
19
|
send_hwm: nil, recv_hwm: nil,
|
|
@@ -42,7 +42,7 @@ module OMQ
|
|
|
42
42
|
# @param recv_hwm [Integer, nil] receive high water mark
|
|
43
43
|
# @param send_timeout [Numeric, nil] send timeout in seconds
|
|
44
44
|
# @param recv_timeout [Numeric, nil] receive timeout in seconds
|
|
45
|
-
# @param backend [Symbol, nil] :ruby
|
|
45
|
+
# @param backend [Symbol, nil] registered backend name (:ruby by default)
|
|
46
46
|
#
|
|
47
47
|
def initialize(endpoints = nil, linger: Float::INFINITY,
|
|
48
48
|
send_hwm: nil, recv_hwm: nil,
|
data/lib/omq/routing/fan_out.rb
CHANGED
|
@@ -177,12 +177,13 @@ module OMQ
|
|
|
177
177
|
#
|
|
178
178
|
def start_conn_send_pump_normal(conn, q, use_wire)
|
|
179
179
|
@engine.spawn_conn_pump_task(conn, annotation: "send pump") do
|
|
180
|
-
batch
|
|
180
|
+
batch = []
|
|
181
|
+
aux_batch = []
|
|
181
182
|
|
|
182
183
|
loop do
|
|
183
184
|
Routing.dequeue_batch(q, batch)
|
|
184
185
|
|
|
185
|
-
if write_matching_batch(conn, batch, use_wire)
|
|
186
|
+
if write_matching_batch(conn, batch, use_wire, aux_batch)
|
|
186
187
|
conn.flush
|
|
187
188
|
batch.each { |parts| @engine.emit_verbose_msg_sent(conn, parts) }
|
|
188
189
|
end
|
|
@@ -197,22 +198,50 @@ module OMQ
|
|
|
197
198
|
#
|
|
198
199
|
# @return [Boolean] true iff at least one message was written
|
|
199
200
|
#
|
|
200
|
-
def write_matching_batch(conn, batch, use_wire)
|
|
201
|
-
|
|
201
|
+
def write_matching_batch(conn, batch, use_wire, aux_batch)
|
|
202
|
+
if use_wire
|
|
203
|
+
write_matching_batch_wire(conn, batch, aux_batch)
|
|
204
|
+
else
|
|
205
|
+
write_matching_batch_framed(conn, batch, aux_batch)
|
|
206
|
+
end
|
|
207
|
+
end
|
|
208
|
+
|
|
202
209
|
|
|
210
|
+
def write_matching_batch_wire(conn, batch, wire_batch)
|
|
203
211
|
batch.each do |parts|
|
|
204
212
|
next unless subscribed?(conn, parts.first || EMPTY_BINARY)
|
|
213
|
+
wire_batch << Protocol::ZMTP::Codec::Frame.encode_message(parts)
|
|
214
|
+
end
|
|
205
215
|
|
|
206
|
-
|
|
207
|
-
|
|
208
|
-
|
|
209
|
-
|
|
210
|
-
|
|
216
|
+
return false if wire_batch.empty?
|
|
217
|
+
|
|
218
|
+
if wire_batch.size == 1
|
|
219
|
+
conn.write_wire(wire_batch.first)
|
|
220
|
+
else
|
|
221
|
+
conn.write_wire_batch(wire_batch)
|
|
222
|
+
end
|
|
211
223
|
|
|
212
|
-
|
|
224
|
+
wire_batch.clear
|
|
225
|
+
true
|
|
226
|
+
end
|
|
227
|
+
|
|
228
|
+
|
|
229
|
+
def write_matching_batch_framed(conn, batch, msg_batch)
|
|
230
|
+
batch.each do |parts|
|
|
231
|
+
next unless subscribed?(conn, parts.first || EMPTY_BINARY)
|
|
232
|
+
msg_batch << parts
|
|
233
|
+
end
|
|
234
|
+
|
|
235
|
+
return false if msg_batch.empty?
|
|
236
|
+
|
|
237
|
+
if msg_batch.size == 1
|
|
238
|
+
conn.write_message(msg_batch.first)
|
|
239
|
+
else
|
|
240
|
+
conn.write_messages(msg_batch)
|
|
213
241
|
end
|
|
214
242
|
|
|
215
|
-
|
|
243
|
+
msg_batch.clear
|
|
244
|
+
true
|
|
216
245
|
end
|
|
217
246
|
|
|
218
247
|
|
data/lib/omq/socket.rb
CHANGED
|
@@ -314,21 +314,31 @@ module OMQ
|
|
|
314
314
|
@options.recv_timeout = recv_timeout if recv_timeout
|
|
315
315
|
@options.conflate = conflate
|
|
316
316
|
@options.on_mute = on_mute if on_mute
|
|
317
|
-
|
|
318
|
-
|
|
319
|
-
|
|
320
|
-
|
|
321
|
-
|
|
322
|
-
|
|
323
|
-
|
|
324
|
-
raise ArgumentError, "unknown backend: #{backend}"
|
|
325
|
-
end
|
|
317
|
+
backend_name = (backend || :ruby).to_sym
|
|
318
|
+
require_backend(backend_name) unless Backend.registered?(backend_name)
|
|
319
|
+
|
|
320
|
+
engine_class = Backend.fetch(backend_name)
|
|
321
|
+
raise ArgumentError, "unknown backend: #{backend}" unless engine_class
|
|
322
|
+
|
|
323
|
+
@engine = engine_class.new(socket_type, @options)
|
|
326
324
|
end
|
|
327
325
|
|
|
328
326
|
|
|
329
327
|
private
|
|
330
328
|
|
|
331
329
|
|
|
330
|
+
def require_backend(name)
|
|
331
|
+
path = case name
|
|
332
|
+
when :libzmq then "omq/backend/libzmq"
|
|
333
|
+
when :ffi then "omq/ffi"
|
|
334
|
+
when :rust then "omq/rust"
|
|
335
|
+
end
|
|
336
|
+
return unless path
|
|
337
|
+
|
|
338
|
+
require path
|
|
339
|
+
end
|
|
340
|
+
|
|
341
|
+
|
|
332
342
|
# Sets the engine's parent task before the first bind or connect.
|
|
333
343
|
# Must be called OUTSIDE Reactor.run so that non-Async callers
|
|
334
344
|
# get the IO thread's root task, not an ephemeral work task.
|
data/lib/omq/transport/inproc.rb
CHANGED
|
@@ -6,16 +6,18 @@ require_relative "inproc/pipe"
|
|
|
6
6
|
|
|
7
7
|
module OMQ
|
|
8
8
|
module Transport
|
|
9
|
-
#
|
|
9
|
+
# Ruby in-process transport (ruby:// scheme).
|
|
10
10
|
#
|
|
11
|
-
# Both peers are Ruby backend sockets in the same process
|
|
12
|
-
#
|
|
13
|
-
# transferred as Ruby arrays — no ZMTP framing, no byte
|
|
11
|
+
# Both peers are Ruby backend sockets in the same process. Messages
|
|
12
|
+
# are transferred as Ruby arrays — no ZMTP framing, no byte
|
|
14
13
|
# serialization. Parts are already frozen by Writable#send, so the
|
|
15
14
|
# receiver sees the same immutable contract as ZMTP transports.
|
|
16
15
|
#
|
|
16
|
+
# The inproc:// scheme is reserved for native backends (FFI/libzmq,
|
|
17
|
+
# Rust/omq-tokio) which have their own in-process registries.
|
|
18
|
+
#
|
|
17
19
|
module Inproc
|
|
18
|
-
Engine.transports["
|
|
20
|
+
Engine.transports["ruby"] = self
|
|
19
21
|
|
|
20
22
|
|
|
21
23
|
# Socket types that exchange commands (SUBSCRIBE/CANCEL) over inproc.
|
|
@@ -38,7 +40,7 @@ module OMQ
|
|
|
38
40
|
|
|
39
41
|
# Creates a bound inproc listener.
|
|
40
42
|
#
|
|
41
|
-
# @param endpoint [String] e.g. "
|
|
43
|
+
# @param endpoint [String] e.g. "ruby://my-endpoint"
|
|
42
44
|
# @param engine [Engine] the owning engine
|
|
43
45
|
# @return [Listener]
|
|
44
46
|
# @raise [ArgumentError] if endpoint is already bound
|
|
@@ -61,7 +63,7 @@ module OMQ
|
|
|
61
63
|
|
|
62
64
|
# Connects to a bound inproc endpoint.
|
|
63
65
|
#
|
|
64
|
-
# @param endpoint [String] e.g. "
|
|
66
|
+
# @param endpoint [String] e.g. "ruby://my-endpoint"
|
|
65
67
|
# @param engine [Engine] the connecting engine
|
|
66
68
|
# @return [void]
|
|
67
69
|
#
|
data/lib/omq/version.rb
CHANGED
data/lib/omq.rb
CHANGED
|
@@ -7,6 +7,7 @@ require "io/stream"
|
|
|
7
7
|
# Core
|
|
8
8
|
require_relative "omq/version"
|
|
9
9
|
require_relative "omq/constants"
|
|
10
|
+
require_relative "omq/backend"
|
|
10
11
|
require_relative "omq/reactor"
|
|
11
12
|
require_relative "omq/options"
|
|
12
13
|
require_relative "omq/routing"
|
|
@@ -24,6 +25,7 @@ require_relative "omq/routing/xsub"
|
|
|
24
25
|
require_relative "omq/routing/push"
|
|
25
26
|
require_relative "omq/routing/pull"
|
|
26
27
|
require_relative "omq/engine"
|
|
28
|
+
OMQ::Backend.register(:ruby, OMQ::Engine)
|
|
27
29
|
|
|
28
30
|
# Transport
|
|
29
31
|
require_relative "omq/transport/inproc"
|