nnq 0.5.0 → 0.6.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/CHANGELOG.md +63 -0
- data/lib/nnq/connection.rb +7 -4
- data/lib/nnq/engine.rb +17 -3
- data/lib/nnq/options.rb +3 -1
- data/lib/nnq/req_rep.rb +51 -11
- data/lib/nnq/routing/backtrace.rb +8 -0
- data/lib/nnq/routing/rep.rb +9 -1
- data/lib/nnq/routing/rep_raw.rb +70 -0
- data/lib/nnq/routing/req.rb +8 -1
- data/lib/nnq/routing/req_raw.rb +80 -0
- data/lib/nnq/routing/respondent.rb +9 -1
- data/lib/nnq/routing/respondent_raw.rb +61 -0
- data/lib/nnq/routing/surveyor.rb +6 -0
- data/lib/nnq/routing/surveyor_raw.rb +113 -0
- data/lib/nnq/socket.rb +4 -3
- data/lib/nnq/surveyor_respondent.rb +38 -18
- data/lib/nnq/version.rb +1 -1
- metadata +7 -3
checksums.yaml
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
SHA256:
|
|
3
|
-
metadata.gz:
|
|
4
|
-
data.tar.gz:
|
|
3
|
+
metadata.gz: 19bceda0339689e8850efe39120590002821f5c859eaf33e4ea1065af317bcaa
|
|
4
|
+
data.tar.gz: 2a15dd79b5cf4f8a41c564e89cf40af7296de8765e8e070de12057b4bc0dedf9
|
|
5
5
|
SHA512:
|
|
6
|
-
metadata.gz:
|
|
7
|
-
data.tar.gz:
|
|
6
|
+
metadata.gz: 1e14a21e6d620df7770c27868f5fdb8ed945fb04137b8817c12fdaa2fdfca1e36a952e6fc4afc5be60562dfd1ff8f43d9d1781ce9a4fb6a1c3a8963cd3738233
|
|
7
|
+
data.tar.gz: 13aae01f03df0b0eca53aa0329d0f2d35b036b4f52744f83033cc46297eae39a60b2c639ad144c9ee3af63cee7d3780c3fff036d30f418e078ccb89253f70761
|
data/CHANGELOG.md
CHANGED
|
@@ -1,5 +1,68 @@
|
|
|
1
1
|
# Changelog
|
|
2
2
|
|
|
3
|
+
## 0.6.1 — 2026-04-15
|
|
4
|
+
|
|
5
|
+
- **Verbose trace (`-vvv`) now fires for cooked REQ/REP/RESPONDENT
|
|
6
|
+
sends.** Cooked `Req#send_request`, `Rep#send_reply`, and
|
|
7
|
+
`Respondent#send_reply` bypass `send_pump` and write to the
|
|
8
|
+
connection directly, so they were never emitting `:message_sent`
|
|
9
|
+
monitor events — `-vvv` only ever showed the `<<` recv side. Each
|
|
10
|
+
now calls `emit_verbose_msg_sent(body)` after the write. Raw
|
|
11
|
+
REQ/REP/RESPONDENT sends get the same treatment (raw surveyor
|
|
12
|
+
already emitted via its per-peer send pump).
|
|
13
|
+
- **Verbose recv previews strip the SP backtrace header.** The recv
|
|
14
|
+
loop used to emit the raw wire body, so `-vvv` traces for
|
|
15
|
+
REQ/REP/SURVEYOR/RESPONDENT showed the 4-byte request/survey id
|
|
16
|
+
(or a multi-word backtrace stack) in front of the payload. Routing
|
|
17
|
+
strategies now expose an optional `preview_body(wire)` hook; the
|
|
18
|
+
engine calls it before emitting `:message_received` so the trace
|
|
19
|
+
shows just the payload.
|
|
20
|
+
- **`Engine#close` drains the monitor queue before cancelling
|
|
21
|
+
tasks.** The monitor consumer fiber lives under the socket-level
|
|
22
|
+
barrier, so `barrier.stop` used to `Async::Stop` it before it had
|
|
23
|
+
a chance to drain trailing events. `close` now emits `:closed`,
|
|
24
|
+
enqueues the nil sentinel, and awaits the stored `monitor_task`
|
|
25
|
+
before stopping the barrier. Fixes flaky `-vvv` traces on
|
|
26
|
+
short-lived sockets where the last `:message_received` event
|
|
27
|
+
would occasionally be lost.
|
|
28
|
+
|
|
29
|
+
## 0.6.0 — 2026-04-15
|
|
30
|
+
|
|
31
|
+
- **NNG-style raw mode for REQ/REP and SURVEYOR/RESPONDENT.** Constructing
|
|
32
|
+
any of the four with `raw: true` bypasses the cooked state machine
|
|
33
|
+
(request-id tracking, pending-reply slot, survey window) and exposes
|
|
34
|
+
the full SP backtrace header as an opaque, caller-supplied handle.
|
|
35
|
+
- `#receive` returns `[pipe, header, body]` where `pipe` is the live
|
|
36
|
+
`NNQ::Connection` that delivered the message (idiomatic Ruby handle
|
|
37
|
+
— no opaque pipe_id token, no lookup registry), `header` is the
|
|
38
|
+
parsed backtrace bytes, and `body` is the payload.
|
|
39
|
+
- Raw REQ/SURVEYOR send: `send(body, header:)` — fans round-robin /
|
|
40
|
+
fans out.
|
|
41
|
+
- Raw REP/RESPONDENT send: `send(body, to:, header:)` — routes
|
|
42
|
+
directly to a prior `pipe` with the stored `header` written
|
|
43
|
+
verbatim, so the cooked peer matches the reply. Closed peer or
|
|
44
|
+
over-TTL header → silent drop (matches NNG behavior).
|
|
45
|
+
- Cooked-mode methods (`send_request`, `send_reply`, `send_survey`)
|
|
46
|
+
raise `NNQ::Error` in raw mode and vice versa.
|
|
47
|
+
- Unblocks proxy/device-style use cases (forwarders, request routers)
|
|
48
|
+
without touching the cooked code paths. `lib/nnq/routing/{req,rep,
|
|
49
|
+
surveyor,respondent}_raw.rb` live alongside their cooked siblings;
|
|
50
|
+
`build_routing` branches on `@raw` inside REQ0/REP0/SURVEYOR0/
|
|
51
|
+
RESPONDENT0. PUB/SUB and PUSH/PULL raw are still out of scope.
|
|
52
|
+
- **Zero-alloc cooked send paths via protocol-sp `header:` kwarg.**
|
|
53
|
+
`Connection#send_message` / `#write_message` grow an optional
|
|
54
|
+
`header:` kwarg that protocol-sp writes between the SP length prefix
|
|
55
|
+
and the body as a third buffered write (coalesced into a single
|
|
56
|
+
`writev`). Cooked `Req#send_request`, `Rep#send_reply`, and
|
|
57
|
+
`Respondent#send_reply` no longer allocate the `header + body`
|
|
58
|
+
intermediate String on every send — the savings apply to every
|
|
59
|
+
REQ/REP round trip regardless of whether raw mode is used.
|
|
60
|
+
Requires `protocol-sp >= 0.3`.
|
|
61
|
+
- **`Options#recv_hwm`** — new option, defaults to `Options::DEFAULT_HWM`
|
|
62
|
+
(same as `send_hwm`). Bounds the raw routing strategies' receive
|
|
63
|
+
queues; the cooked paths still use their existing (unbounded) state
|
|
64
|
+
and are unaffected.
|
|
65
|
+
|
|
3
66
|
## 0.5.0 — 2026-04-15
|
|
4
67
|
|
|
5
68
|
- **Send-path freezes the body** — every public send method (PUSH,
|
data/lib/nnq/connection.rb
CHANGED
|
@@ -35,10 +35,12 @@ module NNQ
|
|
|
35
35
|
# Writes one message into the SP connection's send buffer (no flush).
|
|
36
36
|
#
|
|
37
37
|
# @param body [String]
|
|
38
|
+
# @param header [String, nil] optional binary prefix written between
|
|
39
|
+
# the SP length prefix and body (see Protocol::SP::Connection)
|
|
38
40
|
# @return [void]
|
|
39
|
-
def write_message(body)
|
|
41
|
+
def write_message(body, header: nil)
|
|
40
42
|
raise ClosedError, "connection closed" if @closed
|
|
41
|
-
@sp.write_message(body)
|
|
43
|
+
@sp.write_message(body, header: header)
|
|
42
44
|
end
|
|
43
45
|
|
|
44
46
|
|
|
@@ -57,10 +59,11 @@ module NNQ
|
|
|
57
59
|
# each call is request-paced and there's nothing to batch.
|
|
58
60
|
#
|
|
59
61
|
# @param body [String]
|
|
62
|
+
# @param header [String, nil] optional binary prefix
|
|
60
63
|
# @return [void]
|
|
61
|
-
def send_message(body)
|
|
64
|
+
def send_message(body, header: nil)
|
|
62
65
|
raise ClosedError, "connection closed" if @closed
|
|
63
|
-
@sp.send_message(body)
|
|
66
|
+
@sp.send_message(body, header: header)
|
|
64
67
|
end
|
|
65
68
|
|
|
66
69
|
|
data/lib/nnq/engine.rb
CHANGED
|
@@ -70,6 +70,10 @@ module NNQ
|
|
|
70
70
|
attr_accessor :monitor_queue
|
|
71
71
|
|
|
72
72
|
|
|
73
|
+
# @return [Async::Task, nil] the monitor consumer task, if any
|
|
74
|
+
attr_accessor :monitor_task
|
|
75
|
+
|
|
76
|
+
|
|
73
77
|
# @return [Boolean] when true, {#emit_verbose_monitor_event} forwards
|
|
74
78
|
# per-message traces (:message_sent / :message_received) to the
|
|
75
79
|
# monitor queue. Set by {Socket#monitor} via its +verbose:+ kwarg.
|
|
@@ -286,6 +290,15 @@ module NNQ
|
|
|
286
290
|
# collection mutates during iteration, so snapshot the values.
|
|
287
291
|
@connections.values.each(&:close!)
|
|
288
292
|
|
|
293
|
+
# Emit :closed, seal the monitor queue, and wait for the monitor
|
|
294
|
+
# fiber to drain it before cancelling tasks. Without this join,
|
|
295
|
+
# trailing :message_received events that the recv pump enqueued
|
|
296
|
+
# just before close would be lost when the barrier.stop below
|
|
297
|
+
# Async::Stops the monitor fiber mid-dequeue.
|
|
298
|
+
emit_monitor_event(:closed)
|
|
299
|
+
close_monitor_queue
|
|
300
|
+
@monitor_task&.wait
|
|
301
|
+
|
|
289
302
|
# Cascade-cancel every remaining task (reconnect loops, accept
|
|
290
303
|
# loops, supervisors) in one shot.
|
|
291
304
|
@lifecycle.barrier&.stop
|
|
@@ -295,8 +308,6 @@ module NNQ
|
|
|
295
308
|
# Unblock anyone waiting on peer_connected when the socket is
|
|
296
309
|
# closed before a peer ever arrived.
|
|
297
310
|
@lifecycle.peer_connected.resolve(nil) unless @lifecycle.peer_connected.resolved?
|
|
298
|
-
emit_monitor_event(:closed)
|
|
299
|
-
close_monitor_queue
|
|
300
311
|
end
|
|
301
312
|
|
|
302
313
|
|
|
@@ -337,7 +348,10 @@ module NNQ
|
|
|
337
348
|
@connections[conn].barrier.async(annotation: "nnq recv #{conn.endpoint}") do
|
|
338
349
|
loop do
|
|
339
350
|
body = conn.receive_message
|
|
340
|
-
|
|
351
|
+
if @verbose_monitor
|
|
352
|
+
preview = @routing.respond_to?(:preview_body) ? @routing.preview_body(body) : body
|
|
353
|
+
emit_verbose_msg_received(preview)
|
|
354
|
+
end
|
|
341
355
|
@routing.enqueue(body, conn)
|
|
342
356
|
rescue *CONNECTION_LOST, Async::Stop
|
|
343
357
|
break
|
data/lib/nnq/options.rb
CHANGED
|
@@ -18,19 +18,21 @@ module NNQ
|
|
|
18
18
|
attr_accessor :reconnect_interval
|
|
19
19
|
attr_accessor :max_message_size
|
|
20
20
|
attr_accessor :send_hwm
|
|
21
|
+
attr_accessor :recv_hwm
|
|
21
22
|
attr_accessor :survey_time
|
|
22
23
|
|
|
23
24
|
|
|
24
25
|
# @param linger [Numeric] linger period in seconds on close
|
|
25
26
|
# (default Float::INFINITY = wait forever, matching libzmq).
|
|
26
27
|
# Pass 0 for immediate drop-on-close.
|
|
27
|
-
def initialize(linger: Float::INFINITY, send_hwm: DEFAULT_HWM)
|
|
28
|
+
def initialize(linger: Float::INFINITY, send_hwm: DEFAULT_HWM, recv_hwm: DEFAULT_HWM)
|
|
28
29
|
@linger = linger
|
|
29
30
|
@read_timeout = nil
|
|
30
31
|
@write_timeout = nil
|
|
31
32
|
@reconnect_interval = 0.1
|
|
32
33
|
@max_message_size = nil
|
|
33
34
|
@send_hwm = send_hwm
|
|
35
|
+
@recv_hwm = recv_hwm
|
|
34
36
|
@survey_time = 1.0
|
|
35
37
|
end
|
|
36
38
|
|
data/lib/nnq/req_rep.rb
CHANGED
|
@@ -3,21 +3,45 @@
|
|
|
3
3
|
require_relative "socket"
|
|
4
4
|
require_relative "routing/req"
|
|
5
5
|
require_relative "routing/rep"
|
|
6
|
+
require_relative "routing/req_raw"
|
|
7
|
+
require_relative "routing/rep_raw"
|
|
6
8
|
|
|
7
9
|
module NNQ
|
|
8
|
-
# REQ (nng req0): client side of request/reply.
|
|
9
|
-
# request
|
|
10
|
-
#
|
|
10
|
+
# REQ (nng req0): client side of request/reply. Cooked mode keeps a
|
|
11
|
+
# single in-flight request and matches replies by id; raw mode bypasses
|
|
12
|
+
# the state machine entirely and delivers replies as
|
|
13
|
+
# `[pipe, header, body]` tuples so the app can correlate verbatim.
|
|
11
14
|
#
|
|
12
15
|
class REQ0 < Socket
|
|
13
|
-
#
|
|
14
|
-
# arrives. Returns the reply body (without the id header).
|
|
16
|
+
# Cooked: sends +body+ as a request, blocks until the matching reply
|
|
17
|
+
# arrives. Returns the reply body (without the id header). Raises in
|
|
18
|
+
# raw mode — use {#send} / {#receive} there.
|
|
15
19
|
def send_request(body)
|
|
20
|
+
raise Error, "REQ#send_request not available in raw mode" if raw?
|
|
16
21
|
body = frozen_binary(body)
|
|
17
22
|
Reactor.run { @engine.routing.send_request(body) }
|
|
18
23
|
end
|
|
19
24
|
|
|
20
25
|
|
|
26
|
+
# Raw: round-robins +body+ to the next connected peer with
|
|
27
|
+
# +header+ (typically `[id | 0x80000000].pack("N")`) written
|
|
28
|
+
# verbatim between the SP length prefix and the body. Raises in
|
|
29
|
+
# cooked mode.
|
|
30
|
+
def send(body, header:)
|
|
31
|
+
raise Error, "REQ#send not available in cooked mode" unless raw?
|
|
32
|
+
body = frozen_binary(body)
|
|
33
|
+
Reactor.run { @engine.routing.send(body, header: header) }
|
|
34
|
+
end
|
|
35
|
+
|
|
36
|
+
|
|
37
|
+
# Raw: blocks until the next reply arrives, returns
|
|
38
|
+
# `[pipe, header, body]`. Raises in cooked mode.
|
|
39
|
+
def receive
|
|
40
|
+
raise Error, "REQ#receive not available in cooked mode" unless raw?
|
|
41
|
+
Reactor.run { @engine.routing.receive }
|
|
42
|
+
end
|
|
43
|
+
|
|
44
|
+
|
|
21
45
|
private
|
|
22
46
|
|
|
23
47
|
|
|
@@ -27,28 +51,44 @@ module NNQ
|
|
|
27
51
|
|
|
28
52
|
|
|
29
53
|
def build_routing(engine)
|
|
30
|
-
Routing::Req.new(engine)
|
|
54
|
+
raw? ? Routing::ReqRaw.new(engine) : Routing::Req.new(engine)
|
|
31
55
|
end
|
|
32
56
|
end
|
|
33
57
|
|
|
34
58
|
|
|
35
|
-
# REP (nng rep0): server side of request/reply.
|
|
36
|
-
#
|
|
59
|
+
# REP (nng rep0): server side of request/reply. Cooked mode strictly
|
|
60
|
+
# alternates #receive / #send_reply and stashes the backtrace
|
|
61
|
+
# internally; raw mode exposes the backtrace as an opaque +header+ and
|
|
62
|
+
# the originating pipe as a live Connection, so the app can drive the
|
|
63
|
+
# reply protocol itself (e.g. proxy/device use cases).
|
|
37
64
|
#
|
|
38
65
|
class REP0 < Socket
|
|
39
|
-
#
|
|
66
|
+
# Cooked: returns the next request body. Raw: returns
|
|
67
|
+
# `[pipe, header, body]`.
|
|
40
68
|
def receive
|
|
41
69
|
Reactor.run { @engine.routing.receive }
|
|
42
70
|
end
|
|
43
71
|
|
|
44
72
|
|
|
45
|
-
#
|
|
73
|
+
# Cooked: routes +body+ back to the pipe the most recent #receive
|
|
74
|
+
# came from. Raises in raw mode.
|
|
46
75
|
def send_reply(body)
|
|
76
|
+
raise Error, "REP#send_reply not available in raw mode" if raw?
|
|
47
77
|
body = frozen_binary(body)
|
|
48
78
|
Reactor.run { @engine.routing.send_reply(body) }
|
|
49
79
|
end
|
|
50
80
|
|
|
51
81
|
|
|
82
|
+
# Raw: writes +body+ with +header+ (the opaque backtrace handed out
|
|
83
|
+
# by a prior #receive) back to +to+ (the Connection from the same
|
|
84
|
+
# tuple). Silent drop if +to+ is closed. Raises in cooked mode.
|
|
85
|
+
def send(body, to:, header:)
|
|
86
|
+
raise Error, "REP#send not available in cooked mode" unless raw?
|
|
87
|
+
body = frozen_binary(body)
|
|
88
|
+
Reactor.run { @engine.routing.send(body, to: to, header: header) }
|
|
89
|
+
end
|
|
90
|
+
|
|
91
|
+
|
|
52
92
|
private
|
|
53
93
|
|
|
54
94
|
|
|
@@ -58,7 +98,7 @@ module NNQ
|
|
|
58
98
|
|
|
59
99
|
|
|
60
100
|
def build_routing(engine)
|
|
61
|
-
Routing::Rep.new(engine)
|
|
101
|
+
raw? ? Routing::RepRaw.new(engine) : Routing::Rep.new(engine)
|
|
62
102
|
end
|
|
63
103
|
end
|
|
64
104
|
|
|
@@ -34,6 +34,14 @@ module NNQ
|
|
|
34
34
|
nil # exceeded TTL without finding terminator
|
|
35
35
|
end
|
|
36
36
|
|
|
37
|
+
|
|
38
|
+
# Raw-mode TTL check: returns true if +header+ contains at least
|
|
39
|
+
# MAX_HOPS 4-byte words (i.e. forwarding it would push total hops
|
|
40
|
+
# over the cap). Cheap: just bytesize arithmetic.
|
|
41
|
+
def self.too_many_hops?(header)
|
|
42
|
+
header.bytesize >= MAX_HOPS * 4
|
|
43
|
+
end
|
|
44
|
+
|
|
37
45
|
end
|
|
38
46
|
end
|
|
39
47
|
end
|
data/lib/nnq/routing/rep.rb
CHANGED
|
@@ -69,7 +69,15 @@ module NNQ
|
|
|
69
69
|
end
|
|
70
70
|
|
|
71
71
|
return if conn.closed?
|
|
72
|
-
conn.send_message(
|
|
72
|
+
conn.send_message(body, header: btrace)
|
|
73
|
+
@engine.emit_verbose_msg_sent(body)
|
|
74
|
+
end
|
|
75
|
+
|
|
76
|
+
|
|
77
|
+
# Strips the backtrace header for verbose trace previews.
|
|
78
|
+
def preview_body(wire)
|
|
79
|
+
_, payload = parse_backtrace(wire)
|
|
80
|
+
payload || wire
|
|
73
81
|
end
|
|
74
82
|
|
|
75
83
|
|
|
@@ -0,0 +1,70 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
require "async/limited_queue"
|
|
4
|
+
require_relative "backtrace"
|
|
5
|
+
|
|
6
|
+
module NNQ
|
|
7
|
+
module Routing
|
|
8
|
+
# Raw REP: bypasses the cooked state machine. The incoming
|
|
9
|
+
# backtrace header is split off once at parse time and handed to
|
|
10
|
+
# the caller alongside the live Connection as `[pipe, header, body]`.
|
|
11
|
+
# Replies go back via `send(body, to:, header:)` which writes the
|
|
12
|
+
# caller-supplied header verbatim — no cooked pending/echo logic,
|
|
13
|
+
# no single-in-flight constraint.
|
|
14
|
+
#
|
|
15
|
+
class RepRaw
|
|
16
|
+
include Backtrace
|
|
17
|
+
|
|
18
|
+
|
|
19
|
+
def initialize(engine)
|
|
20
|
+
@engine = engine
|
|
21
|
+
@recv_queue = Async::LimitedQueue.new(engine.options.recv_hwm)
|
|
22
|
+
end
|
|
23
|
+
|
|
24
|
+
|
|
25
|
+
# @return [Array, nil] [pipe, header, body] or nil on close
|
|
26
|
+
def receive
|
|
27
|
+
@recv_queue.dequeue
|
|
28
|
+
end
|
|
29
|
+
|
|
30
|
+
|
|
31
|
+
# Sends +body+ with the caller-supplied +header+ back to +to+
|
|
32
|
+
# (a Connection handed out by a prior #receive). Silent drop
|
|
33
|
+
# if the target is closed or the header would push total hops
|
|
34
|
+
# over MAX_HOPS.
|
|
35
|
+
def send(body, to:, header:)
|
|
36
|
+
return if to.closed?
|
|
37
|
+
return if Backtrace.too_many_hops?(header)
|
|
38
|
+
to.send_message(body, header: header)
|
|
39
|
+
@engine.emit_verbose_msg_sent(body)
|
|
40
|
+
rescue ClosedError
|
|
41
|
+
# peer went away between receive and send — drop
|
|
42
|
+
end
|
|
43
|
+
|
|
44
|
+
|
|
45
|
+
def preview_body(wire)
|
|
46
|
+
_, payload = parse_backtrace(wire)
|
|
47
|
+
payload || wire
|
|
48
|
+
end
|
|
49
|
+
|
|
50
|
+
|
|
51
|
+
# Called by the engine recv loop.
|
|
52
|
+
def enqueue(wire_bytes, conn)
|
|
53
|
+
header, payload = parse_backtrace(wire_bytes)
|
|
54
|
+
return unless header # malformed / over-TTL — drop
|
|
55
|
+
@recv_queue.enqueue([conn, header, payload])
|
|
56
|
+
end
|
|
57
|
+
|
|
58
|
+
|
|
59
|
+
def close
|
|
60
|
+
@recv_queue.enqueue(nil)
|
|
61
|
+
end
|
|
62
|
+
|
|
63
|
+
|
|
64
|
+
def close_read
|
|
65
|
+
@recv_queue.enqueue(nil)
|
|
66
|
+
end
|
|
67
|
+
|
|
68
|
+
end
|
|
69
|
+
end
|
|
70
|
+
end
|
data/lib/nnq/routing/req.rb
CHANGED
|
@@ -55,7 +55,8 @@ module NNQ
|
|
|
55
55
|
|
|
56
56
|
conn = pick_peer
|
|
57
57
|
header = [id].pack("N")
|
|
58
|
-
conn.send_message(header
|
|
58
|
+
conn.send_message(body, header: header)
|
|
59
|
+
@engine.emit_verbose_msg_sent(body)
|
|
59
60
|
promise.wait
|
|
60
61
|
ensure
|
|
61
62
|
@mutex.synchronize do
|
|
@@ -81,6 +82,12 @@ module NNQ
|
|
|
81
82
|
end
|
|
82
83
|
|
|
83
84
|
|
|
85
|
+
# Strips the 4-byte request id for verbose trace previews.
|
|
86
|
+
def preview_body(wire)
|
|
87
|
+
wire.byteslice(4..) || wire
|
|
88
|
+
end
|
|
89
|
+
|
|
90
|
+
|
|
84
91
|
def close
|
|
85
92
|
@mutex.synchronize do
|
|
86
93
|
@outstanding&.last&.reject(NNQ::Error.new("REQ socket closed"))
|
|
@@ -0,0 +1,80 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
require "async/limited_queue"
|
|
4
|
+
require_relative "backtrace"
|
|
5
|
+
|
|
6
|
+
module NNQ
|
|
7
|
+
module Routing
|
|
8
|
+
# Raw REQ: bypasses the cooked single-in-flight request-id
|
|
9
|
+
# state machine. Sends are fire-and-forget round-robin with a
|
|
10
|
+
# caller-supplied header (typically `[id | 0x80000000].pack("N")`);
|
|
11
|
+
# replies land in a bounded queue and are delivered as
|
|
12
|
+
# `[pipe, header, body]` tuples so the app can correlate by
|
|
13
|
+
# header verbatim without ever parsing or slicing bytes.
|
|
14
|
+
#
|
|
15
|
+
class ReqRaw
|
|
16
|
+
include Backtrace
|
|
17
|
+
|
|
18
|
+
|
|
19
|
+
def initialize(engine)
|
|
20
|
+
@engine = engine
|
|
21
|
+
@next_idx = 0
|
|
22
|
+
@recv_queue = Async::LimitedQueue.new(engine.options.recv_hwm)
|
|
23
|
+
end
|
|
24
|
+
|
|
25
|
+
|
|
26
|
+
def send(body, header:)
|
|
27
|
+
conn = pick_peer
|
|
28
|
+
conn.send_message(body, header: header)
|
|
29
|
+
@engine.emit_verbose_msg_sent(body)
|
|
30
|
+
end
|
|
31
|
+
|
|
32
|
+
|
|
33
|
+
def preview_body(wire)
|
|
34
|
+
_, payload = parse_backtrace(wire)
|
|
35
|
+
payload || wire
|
|
36
|
+
end
|
|
37
|
+
|
|
38
|
+
|
|
39
|
+
def receive
|
|
40
|
+
@recv_queue.dequeue
|
|
41
|
+
end
|
|
42
|
+
|
|
43
|
+
|
|
44
|
+
def enqueue(wire_bytes, conn)
|
|
45
|
+
header, payload = parse_backtrace(wire_bytes)
|
|
46
|
+
return unless header
|
|
47
|
+
@recv_queue.enqueue([conn, header, payload])
|
|
48
|
+
end
|
|
49
|
+
|
|
50
|
+
|
|
51
|
+
def close
|
|
52
|
+
@recv_queue.enqueue(nil)
|
|
53
|
+
end
|
|
54
|
+
|
|
55
|
+
|
|
56
|
+
def close_read
|
|
57
|
+
@recv_queue.enqueue(nil)
|
|
58
|
+
end
|
|
59
|
+
|
|
60
|
+
|
|
61
|
+
private
|
|
62
|
+
|
|
63
|
+
|
|
64
|
+
def pick_peer
|
|
65
|
+
loop do
|
|
66
|
+
conns = @engine.connections.keys
|
|
67
|
+
|
|
68
|
+
if conns.empty?
|
|
69
|
+
@engine.new_pipe.wait
|
|
70
|
+
next
|
|
71
|
+
end
|
|
72
|
+
|
|
73
|
+
@next_idx = (@next_idx + 1) % conns.size
|
|
74
|
+
return conns[@next_idx]
|
|
75
|
+
end
|
|
76
|
+
end
|
|
77
|
+
|
|
78
|
+
end
|
|
79
|
+
end
|
|
80
|
+
end
|
|
@@ -51,7 +51,15 @@ module NNQ
|
|
|
51
51
|
end
|
|
52
52
|
|
|
53
53
|
return if conn.closed?
|
|
54
|
-
conn.send_message(
|
|
54
|
+
conn.send_message(body, header: btrace)
|
|
55
|
+
@engine.emit_verbose_msg_sent(body)
|
|
56
|
+
end
|
|
57
|
+
|
|
58
|
+
|
|
59
|
+
# Strips the backtrace header for verbose trace previews.
|
|
60
|
+
def preview_body(wire)
|
|
61
|
+
_, payload = parse_backtrace(wire)
|
|
62
|
+
payload || wire
|
|
55
63
|
end
|
|
56
64
|
|
|
57
65
|
|
|
@@ -0,0 +1,61 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
require "async/limited_queue"
|
|
4
|
+
require_relative "backtrace"
|
|
5
|
+
|
|
6
|
+
module NNQ
|
|
7
|
+
module Routing
|
|
8
|
+
# Raw RESPONDENT: mirror of {RepRaw} for the survey pattern.
|
|
9
|
+
# No survey-window state, no pending slot — the app receives
|
|
10
|
+
# `[pipe, header, body]` tuples and chooses whether (and when)
|
|
11
|
+
# to reply via `send(body, to:, header:)`.
|
|
12
|
+
#
|
|
13
|
+
class RespondentRaw
|
|
14
|
+
include Backtrace
|
|
15
|
+
|
|
16
|
+
|
|
17
|
+
def initialize(engine)
|
|
18
|
+
@engine = engine
|
|
19
|
+
@recv_queue = Async::LimitedQueue.new(engine.options.recv_hwm)
|
|
20
|
+
end
|
|
21
|
+
|
|
22
|
+
|
|
23
|
+
def receive
|
|
24
|
+
@recv_queue.dequeue
|
|
25
|
+
end
|
|
26
|
+
|
|
27
|
+
|
|
28
|
+
def send(body, to:, header:)
|
|
29
|
+
return if to.closed?
|
|
30
|
+
return if Backtrace.too_many_hops?(header)
|
|
31
|
+
to.send_message(body, header: header)
|
|
32
|
+
@engine.emit_verbose_msg_sent(body)
|
|
33
|
+
rescue ClosedError
|
|
34
|
+
end
|
|
35
|
+
|
|
36
|
+
|
|
37
|
+
def preview_body(wire)
|
|
38
|
+
_, payload = parse_backtrace(wire)
|
|
39
|
+
payload || wire
|
|
40
|
+
end
|
|
41
|
+
|
|
42
|
+
|
|
43
|
+
def enqueue(wire_bytes, conn)
|
|
44
|
+
header, payload = parse_backtrace(wire_bytes)
|
|
45
|
+
return unless header
|
|
46
|
+
@recv_queue.enqueue([conn, header, payload])
|
|
47
|
+
end
|
|
48
|
+
|
|
49
|
+
|
|
50
|
+
def close
|
|
51
|
+
@recv_queue.enqueue(nil)
|
|
52
|
+
end
|
|
53
|
+
|
|
54
|
+
|
|
55
|
+
def close_read
|
|
56
|
+
@recv_queue.enqueue(nil)
|
|
57
|
+
end
|
|
58
|
+
|
|
59
|
+
end
|
|
60
|
+
end
|
|
61
|
+
end
|
data/lib/nnq/routing/surveyor.rb
CHANGED
|
@@ -78,6 +78,12 @@ module NNQ
|
|
|
78
78
|
end
|
|
79
79
|
|
|
80
80
|
|
|
81
|
+
# Strips the 4-byte survey id for verbose trace previews.
|
|
82
|
+
def preview_body(wire)
|
|
83
|
+
wire.byteslice(4..) || wire
|
|
84
|
+
end
|
|
85
|
+
|
|
86
|
+
|
|
81
87
|
def connection_added(conn)
|
|
82
88
|
queue = Async::LimitedQueue.new(@engine.options.send_hwm)
|
|
83
89
|
@queues[conn] = queue
|
|
@@ -0,0 +1,113 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
require "async"
|
|
4
|
+
require "async/limited_queue"
|
|
5
|
+
require_relative "backtrace"
|
|
6
|
+
|
|
7
|
+
module NNQ
|
|
8
|
+
module Routing
|
|
9
|
+
# Raw SURVEYOR: fans out surveys to all peers like cooked
|
|
10
|
+
# {Surveyor}, but without a survey window, survey-id matching,
|
|
11
|
+
# or timeout. Replies are delivered as `[pipe, header, body]`
|
|
12
|
+
# tuples so the app can correlate by header verbatim.
|
|
13
|
+
#
|
|
14
|
+
# Each per-conn send queue holds `[header, body]` pairs and the
|
|
15
|
+
# pump calls `conn.write_message(body, header: header)` so the
|
|
16
|
+
# protocol-sp header kwarg is threaded through the fan-out —
|
|
17
|
+
# zero concat even on the broadcast path.
|
|
18
|
+
#
|
|
19
|
+
class SurveyorRaw
|
|
20
|
+
include Backtrace
|
|
21
|
+
|
|
22
|
+
|
|
23
|
+
def initialize(engine)
|
|
24
|
+
@engine = engine
|
|
25
|
+
@queues = {} # conn => Async::LimitedQueue
|
|
26
|
+
@pump_tasks = {} # conn => Async::Task
|
|
27
|
+
@recv_queue = Async::LimitedQueue.new(engine.options.recv_hwm)
|
|
28
|
+
end
|
|
29
|
+
|
|
30
|
+
|
|
31
|
+
def send(body, header:)
|
|
32
|
+
@queues.each_value do |q|
|
|
33
|
+
q.enqueue([header, body]) unless q.limited?
|
|
34
|
+
end
|
|
35
|
+
end
|
|
36
|
+
|
|
37
|
+
|
|
38
|
+
def receive
|
|
39
|
+
@recv_queue.dequeue
|
|
40
|
+
end
|
|
41
|
+
|
|
42
|
+
|
|
43
|
+
def enqueue(wire_bytes, conn)
|
|
44
|
+
header, payload = parse_backtrace(wire_bytes)
|
|
45
|
+
return unless header
|
|
46
|
+
@recv_queue.enqueue([conn, header, payload])
|
|
47
|
+
end
|
|
48
|
+
|
|
49
|
+
|
|
50
|
+
def preview_body(wire)
|
|
51
|
+
_, payload = parse_backtrace(wire)
|
|
52
|
+
payload || wire
|
|
53
|
+
end
|
|
54
|
+
|
|
55
|
+
|
|
56
|
+
def connection_added(conn)
|
|
57
|
+
queue = Async::LimitedQueue.new(@engine.options.send_hwm)
|
|
58
|
+
@queues[conn] = queue
|
|
59
|
+
@pump_tasks[conn] = spawn_pump(conn, queue)
|
|
60
|
+
end
|
|
61
|
+
|
|
62
|
+
|
|
63
|
+
def connection_removed(conn)
|
|
64
|
+
@queues.delete(conn)
|
|
65
|
+
task = @pump_tasks.delete(conn)
|
|
66
|
+
|
|
67
|
+
return unless task
|
|
68
|
+
return if task == Async::Task.current
|
|
69
|
+
|
|
70
|
+
task.stop
|
|
71
|
+
rescue IOError, Errno::EPIPE
|
|
72
|
+
end
|
|
73
|
+
|
|
74
|
+
|
|
75
|
+
def send_queue_drained?
|
|
76
|
+
@queues.each_value.all? { |q| q.empty? }
|
|
77
|
+
end
|
|
78
|
+
|
|
79
|
+
|
|
80
|
+
def close
|
|
81
|
+
@pump_tasks.each_value(&:stop)
|
|
82
|
+
@pump_tasks.clear
|
|
83
|
+
@queues.clear
|
|
84
|
+
@recv_queue.enqueue(nil)
|
|
85
|
+
end
|
|
86
|
+
|
|
87
|
+
|
|
88
|
+
def close_read
|
|
89
|
+
@recv_queue.enqueue(nil)
|
|
90
|
+
end
|
|
91
|
+
|
|
92
|
+
|
|
93
|
+
private
|
|
94
|
+
|
|
95
|
+
|
|
96
|
+
def spawn_pump(conn, queue)
|
|
97
|
+
annotation = "nnq surveyor_raw pump #{conn.endpoint}"
|
|
98
|
+
parent = @engine.connections[conn]&.barrier || @engine.barrier
|
|
99
|
+
|
|
100
|
+
@engine.spawn_task(annotation:, parent:) do
|
|
101
|
+
loop do
|
|
102
|
+
header, body = queue.dequeue
|
|
103
|
+
conn.send_message(body, header: header)
|
|
104
|
+
@engine.emit_verbose_msg_sent(body)
|
|
105
|
+
rescue EOFError, IOError, Errno::EPIPE, Errno::ECONNRESET
|
|
106
|
+
break
|
|
107
|
+
end
|
|
108
|
+
end
|
|
109
|
+
end
|
|
110
|
+
|
|
111
|
+
end
|
|
112
|
+
end
|
|
113
|
+
end
|
data/lib/nnq/socket.rb
CHANGED
|
@@ -32,9 +32,9 @@ module NNQ
|
|
|
32
32
|
|
|
33
33
|
# @yieldparam [self] the socket; when a block is passed the socket
|
|
34
34
|
# is {#close}d when the block returns (or raises), File.open-style.
|
|
35
|
-
def initialize(raw: false, linger: Float::INFINITY, send_hwm: Options::DEFAULT_HWM)
|
|
35
|
+
def initialize(raw: false, linger: Float::INFINITY, send_hwm: Options::DEFAULT_HWM, recv_hwm: Options::DEFAULT_HWM)
|
|
36
36
|
@raw = raw
|
|
37
|
-
@options = Options.new(linger: linger, send_hwm: send_hwm)
|
|
37
|
+
@options = Options.new(linger: linger, send_hwm: send_hwm, recv_hwm: recv_hwm)
|
|
38
38
|
@engine = Engine.new(protocol: protocol, options: @options) { |engine| build_routing(engine) }
|
|
39
39
|
|
|
40
40
|
begin
|
|
@@ -126,13 +126,14 @@ module NNQ
|
|
|
126
126
|
@engine.verbose_monitor = verbose
|
|
127
127
|
|
|
128
128
|
Reactor.run do
|
|
129
|
-
@engine.spawn_task(annotation: "nnq monitor") do
|
|
129
|
+
@engine.monitor_task = @engine.spawn_task(annotation: "nnq monitor") do
|
|
130
130
|
while (event = queue.dequeue)
|
|
131
131
|
block.call(event)
|
|
132
132
|
end
|
|
133
133
|
rescue Async::Stop
|
|
134
134
|
ensure
|
|
135
135
|
@engine.monitor_queue = nil
|
|
136
|
+
@engine.monitor_task = nil
|
|
136
137
|
block.call(MonitorEvent.new(type: :monitor_stopped))
|
|
137
138
|
end
|
|
138
139
|
end
|
|
@@ -3,27 +3,37 @@
|
|
|
3
3
|
require_relative "socket"
|
|
4
4
|
require_relative "routing/surveyor"
|
|
5
5
|
require_relative "routing/respondent"
|
|
6
|
+
require_relative "routing/surveyor_raw"
|
|
7
|
+
require_relative "routing/respondent_raw"
|
|
6
8
|
|
|
7
9
|
module NNQ
|
|
8
10
|
# SURVEYOR (nng surveyor0): broadcast side of the survey pattern.
|
|
9
|
-
#
|
|
10
|
-
#
|
|
11
|
-
#
|
|
12
|
-
# Only one outstanding survey at a time — sending a new survey
|
|
13
|
-
# abandons the previous one. Respondents are not obliged to reply.
|
|
11
|
+
# Cooked mode enforces a timed survey window and matches replies by
|
|
12
|
+
# survey id; raw mode fans out with a caller-supplied +header+ and
|
|
13
|
+
# delivers replies as `[pipe, header, body]` with no timer.
|
|
14
14
|
#
|
|
15
15
|
class SURVEYOR0 < Socket
|
|
16
|
-
#
|
|
16
|
+
# Cooked: broadcasts +body+ as a survey to all connected respondents.
|
|
17
17
|
def send_survey(body)
|
|
18
|
+
raise Error, "SURVEYOR#send_survey not available in raw mode" if raw?
|
|
18
19
|
body = frozen_binary(body)
|
|
19
20
|
Reactor.run { @engine.routing.send_survey(body) }
|
|
20
21
|
end
|
|
21
22
|
|
|
22
23
|
|
|
23
|
-
#
|
|
24
|
-
#
|
|
25
|
-
#
|
|
26
|
-
|
|
24
|
+
# Raw: broadcasts +body+ with +header+ to all connected respondents
|
|
25
|
+
# via per-conn send pumps (header is threaded through the
|
|
26
|
+
# protocol-sp header kwarg — no concat). Raises in cooked mode.
|
|
27
|
+
def send(body, header:)
|
|
28
|
+
raise Error, "SURVEYOR#send not available in cooked mode" unless raw?
|
|
29
|
+
body = frozen_binary(body)
|
|
30
|
+
Reactor.run { @engine.routing.send(body, header: header) }
|
|
31
|
+
end
|
|
32
|
+
|
|
33
|
+
|
|
34
|
+
# Cooked: receives the next reply within the survey window, raises
|
|
35
|
+
# {NNQ::TimedOut} on window expiry. Raw: returns `[pipe, header, body]`
|
|
36
|
+
# and blocks indefinitely (no survey window).
|
|
27
37
|
def receive
|
|
28
38
|
Reactor.run { @engine.routing.receive }
|
|
29
39
|
end
|
|
@@ -38,31 +48,41 @@ module NNQ
|
|
|
38
48
|
|
|
39
49
|
|
|
40
50
|
def build_routing(engine)
|
|
41
|
-
Routing::Surveyor.new(engine)
|
|
51
|
+
raw? ? Routing::SurveyorRaw.new(engine) : Routing::Surveyor.new(engine)
|
|
42
52
|
end
|
|
43
53
|
end
|
|
44
54
|
|
|
45
55
|
|
|
46
56
|
# RESPONDENT (nng respondent0): reply side of the survey pattern.
|
|
47
|
-
#
|
|
48
|
-
#
|
|
57
|
+
# Cooked mode strictly alternates #receive / #send_reply; raw mode
|
|
58
|
+
# exposes the backtrace as an opaque +header+ and the originating
|
|
59
|
+
# surveyor pipe as a live Connection.
|
|
49
60
|
#
|
|
50
61
|
class RESPONDENT0 < Socket
|
|
51
|
-
#
|
|
52
|
-
#
|
|
53
|
-
# @return [String, nil] survey body, or nil if the socket was closed
|
|
62
|
+
# Cooked: blocks until the next survey arrives. Raw: returns
|
|
63
|
+
# `[pipe, header, body]`.
|
|
54
64
|
def receive
|
|
55
65
|
Reactor.run { @engine.routing.receive }
|
|
56
66
|
end
|
|
57
67
|
|
|
58
68
|
|
|
59
|
-
#
|
|
69
|
+
# Cooked: routes +body+ back to the surveyor that sent the most
|
|
70
|
+
# recent survey. Raises in raw mode.
|
|
60
71
|
def send_reply(body)
|
|
72
|
+
raise Error, "RESPONDENT#send_reply not available in raw mode" if raw?
|
|
61
73
|
body = frozen_binary(body)
|
|
62
74
|
Reactor.run { @engine.routing.send_reply(body) }
|
|
63
75
|
end
|
|
64
76
|
|
|
65
77
|
|
|
78
|
+
# Raw: writes +body+ with +header+ back to +to+. Raises in cooked mode.
|
|
79
|
+
def send(body, to:, header:)
|
|
80
|
+
raise Error, "RESPONDENT#send not available in cooked mode" unless raw?
|
|
81
|
+
body = frozen_binary(body)
|
|
82
|
+
Reactor.run { @engine.routing.send(body, to: to, header: header) }
|
|
83
|
+
end
|
|
84
|
+
|
|
85
|
+
|
|
66
86
|
private
|
|
67
87
|
|
|
68
88
|
|
|
@@ -72,7 +92,7 @@ module NNQ
|
|
|
72
92
|
|
|
73
93
|
|
|
74
94
|
def build_routing(engine)
|
|
75
|
-
Routing::Respondent.new(engine)
|
|
95
|
+
raw? ? Routing::RespondentRaw.new(engine) : Routing::Respondent.new(engine)
|
|
76
96
|
end
|
|
77
97
|
end
|
|
78
98
|
end
|
data/lib/nnq/version.rb
CHANGED
metadata
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
|
2
2
|
name: nnq
|
|
3
3
|
version: !ruby/object:Gem::Version
|
|
4
|
-
version: 0.
|
|
4
|
+
version: 0.6.1
|
|
5
5
|
platform: ruby
|
|
6
6
|
authors:
|
|
7
7
|
- Patrik Wenger
|
|
@@ -43,14 +43,14 @@ dependencies:
|
|
|
43
43
|
requirements:
|
|
44
44
|
- - ">="
|
|
45
45
|
- !ruby/object:Gem::Version
|
|
46
|
-
version: '0.
|
|
46
|
+
version: '0.3'
|
|
47
47
|
type: :runtime
|
|
48
48
|
prerelease: false
|
|
49
49
|
version_requirements: !ruby/object:Gem::Requirement
|
|
50
50
|
requirements:
|
|
51
51
|
- - ">="
|
|
52
52
|
- !ruby/object:Gem::Version
|
|
53
|
-
version: '0.
|
|
53
|
+
version: '0.3'
|
|
54
54
|
description: Pure Ruby implementation of nanomsg's Scalability Protocols (SP) on top
|
|
55
55
|
of async + io-stream. Per-socket HWM, opportunistic batching, wire-compatible with
|
|
56
56
|
libnng over inproc/ipc/tcp.
|
|
@@ -85,11 +85,15 @@ files:
|
|
|
85
85
|
- lib/nnq/routing/pull.rb
|
|
86
86
|
- lib/nnq/routing/push.rb
|
|
87
87
|
- lib/nnq/routing/rep.rb
|
|
88
|
+
- lib/nnq/routing/rep_raw.rb
|
|
88
89
|
- lib/nnq/routing/req.rb
|
|
90
|
+
- lib/nnq/routing/req_raw.rb
|
|
89
91
|
- lib/nnq/routing/respondent.rb
|
|
92
|
+
- lib/nnq/routing/respondent_raw.rb
|
|
90
93
|
- lib/nnq/routing/send_pump.rb
|
|
91
94
|
- lib/nnq/routing/sub.rb
|
|
92
95
|
- lib/nnq/routing/surveyor.rb
|
|
96
|
+
- lib/nnq/routing/surveyor_raw.rb
|
|
93
97
|
- lib/nnq/socket.rb
|
|
94
98
|
- lib/nnq/surveyor_respondent.rb
|
|
95
99
|
- lib/nnq/transport/inproc.rb
|