nnq 0.4.0 → 0.6.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: ed71a88238bae0611223d36bb3ec0a39795388d7b90227fb690cfc924cf85198
-  data.tar.gz: cda3bfff65005960b91672cd1c6e2a61e8fce104e202ed928932e7b0404d7eda
+  metadata.gz: 0a336ac1e24bc6210ddeac6731163baab6f8980d9eebbe3235fac13157416fcf
+  data.tar.gz: f049bf9038235487966cae1c8920b452abf9984e3776b2b93cbbd95e067fb485
 SHA512:
-  metadata.gz: 87ddb00a39836f3699dbd5f17a18b34dbd1c0c18d284ad1cf0b14d6271e565444ffce926ce2514c912423f943ef9a994ebb79eb3620b238f872f840124e3ad38
-  data.tar.gz: 193a8a3a1830f18aba6399990a7f1ea81639165c6ffc072691874ca972e21732f4edaf6fff3a82c3bf40b1d3a117ff04adc381e1a3f65d2f2ca677b091ae9d9d
+  metadata.gz: dcef45943a41f1bc53bbcf8ecc0c34c2779e4a16dea04e8f78854cf6a9debe11168596b47b764728e49393f61c319d41e2d79e489b37dec809adc59cbc0741f0
+  data.tar.gz: fd7aaa30c57d5b8fbfd6279aba8b96423e201837ca4ef0144a315ec020da5e0183aaede0c7327fc49bf1bd318894099e4fe02657a12aebf98c1d895582583c62

data/CHANGELOG.md

@@ -1,5 +1,119 @@
 # Changelog
 
+## 0.6.0 — 2026-04-15
+
+- **NNG-style raw mode for REQ/REP and SURVEYOR/RESPONDENT.** Constructing
+  any of the four with `raw: true` bypasses the cooked state machine
+  (request-id tracking, pending-reply slot, survey window) and exposes
+  the full SP backtrace header as an opaque, caller-supplied handle.
+  - `#receive` returns `[pipe, header, body]` where `pipe` is the live
+    `NNQ::Connection` that delivered the message (idiomatic Ruby handle
+    — no opaque pipe_id token, no lookup registry), `header` is the
+    parsed backtrace bytes, and `body` is the payload.
+  - Raw REQ/SURVEYOR send: `send(body, header:)` — fans round-robin /
+    fans out.
+  - Raw REP/RESPONDENT send: `send(body, to:, header:)` — routes
+    directly to a prior `pipe` with the stored `header` written
+    verbatim, so the cooked peer matches the reply. Closed peer or
+    over-TTL header → silent drop (matches NNG behavior).
+  - Cooked-mode methods (`send_request`, `send_reply`, `send_survey`)
+    raise `NNQ::Error` in raw mode and vice versa.
+  - Unblocks proxy/device-style use cases (forwarders, request routers)
+    without touching the cooked code paths. `lib/nnq/routing/{req,rep,
+    surveyor,respondent}_raw.rb` live alongside their cooked siblings;
+    `build_routing` branches on `@raw` inside REQ0/REP0/SURVEYOR0/
+    RESPONDENT0. PUB/SUB and PUSH/PULL raw are still out of scope.
+- **Zero-alloc cooked send paths via protocol-sp `header:` kwarg.**
+  `Connection#send_message` / `#write_message` grow an optional
+  `header:` kwarg that protocol-sp writes between the SP length prefix
+  and the body as a third buffered write (coalesced into a single
+  `writev`). Cooked `Req#send_request`, `Rep#send_reply`, and
+  `Respondent#send_reply` no longer allocate the `header + body`
+  intermediate String on every send — the savings apply to every
+  REQ/REP round trip regardless of whether raw mode is used.
+  Requires `protocol-sp >= 0.3`.
+- **`Options#recv_hwm`** — new option, defaults to `Options::DEFAULT_HWM`
+  (same as `send_hwm`). Bounds the raw routing strategies' receive
+  queues; the cooked paths still use their existing (unbounded) state
+  and are unaffected.
+
+## 0.5.0 — 2026-04-15
+
+- **Send-path freezes the body** — every public send method (PUSH,
+  PUB, PAIR, BUS, REQ, REP, SURVEYOR, RESPONDENT) routes the body
+  through `Socket#frozen_binary`, which coerces to a frozen binary
+  string. Fast path: already frozen and binary → returned as-is, no
+  allocation. Slow path: `body.b.freeze` (one copy). Prevents a
+  caller from mutating the string after it has been enqueued (the
+  body can sit in a send queue or per-peer queue until a pump
+  writes it).
+- **Hot-path: no kwargs splat on verbose monitor emit** —
+  `emit_verbose_monitor_event(type, **detail)` replaced with dedicated
+  `emit_verbose_msg_sent(body)` / `emit_verbose_msg_received(body)`
+  helpers. Early-returns before allocating the detail hash, so the
+  send/recv loops pay nothing when `-vvv` is off. Send pump also
+  hoists the `verbose_monitor` check out of the batch `.each`.
+- **YJIT-friendly `all?` blocks** — `@queues.each_value.all?(&:empty?)`
+  → explicit `{ |q| q.empty? }` in pub/bus/surveyor `drained?`
+  (YJIT specializes explicit blocks, not `Symbol#to_proc`).
+- **`Reactor.run` uses `Async::Promise`** — replaces the
+  `Thread::Queue` + manual `[:ok,val]`/`[:error,exc]` tagging with a
+  single `result.fulfill { block.call }` + `result.wait` pair.
+- **`Engine#spawn_task(parent:)`** — renamed from `barrier:` to make it
+  clear any parent barrier is accepted, not just the socket-level one.
+- **`linger` default → `Float::INFINITY`** — matches libzmq parity.
+  `Socket#close` waits forever for the send queue to drain. Pass
+  `linger: 0` for the old drop-on-close behavior.
+- **`Socket.new` accepts a block** — File.open-style. The socket is
+  yielded to the block and `#close`d when the block returns (or
+  raises).
+- **`drain_send_queue` rescues `Async::Stop`** — parent-task
+  cancellation during close no longer propagates out of the ensure
+  path; the rest of teardown runs.
+- **Hot-path `Array#first`** — `send_pump` uses `Array#first` instead
+  of `[0]` for YJIT specialization.
+- **Barrier-based cascading teardown** — `SocketLifecycle` owns a
+  socket-level `Async::Barrier`; `ConnectionLifecycle` creates a nested
+  per-connection barrier. All pumps, accept loops, reconnect loops, and
+  supervisors live under these barriers. `Engine#close` calls
+  `barrier.stop` once and every descendant unwinds atomically. Replaces
+  the manual `@tasks` array.
+- **Per-connection supervisor** — each connection spawns a supervisor
+  task (on the socket barrier) that watches for the first pump exit and
+  runs `lost!` in `ensure`. Placing the supervisor outside the
+  per-connection barrier avoids the self-stop footgun.
+- **Connect timeout** — `Transport::TCP.connect` uses
+  `Socket.tcp(host, port, connect_timeout:)` instead of `TCPSocket.new`.
+  Timeout derived from `reconnect_interval` (floor 0.5s). Fixes macOS
+  hang where IPv6 `connect(2)` never delivers `ECONNREFUSED`.
+- **Handshake timeout** — SP greeting exchange wrapped in
+  `Async::Task#with_timeout(handshake_timeout)`. Prevents a hang when a
+  non-NNG service accepts the TCP connection but never sends a greeting.
+- **Reconnect after handshake failure** — `ConnectionLifecycle#handshake!`
+  now calls `tear_down!(reconnect: true)` on error instead of bare
+  `transition!(:closed)`, so the endpoint doesn't go dead when a peer
+  RSTs mid-handshake.
+- **Quantized reconnect sleeps** — `Reconnect#quantized_wait` aligns
+  retries to wall-clock grid boundaries. Multiple clients reconnecting
+  with the same interval wake at the same instant.
+- **Send pump fairness yield** — `Async::Task.current.yield` after each
+  batch write ensures peer pumps get a turn when the queue stays
+  non-empty.
+- Add `DESIGN.md` documenting the architecture.
+- **Versioned socket names** — `PUSH` → `PUSH0`, `PULL` → `PULL0`, etc.
+  Canonical names now include the SP protocol version. Unversioned
+  aliases (`NNQ::PUSH = NNQ::PUSH0`) are kept for backward compat.
+- **`raw:` kwarg** — `Socket#initialize` accepts `raw: false`. Plumbing
+  for raw-mode routing (device/proxy support). No functional raw
+  routing yet.
+- **`NNQ::BUS0`** — best-effort bidirectional mesh (bus0). Fan-out send
+  to all peers (drop when full), shared recv queue. Self-pairing.
+- **`NNQ::SURVEYOR0` / `NNQ::RESPONDENT0`** — survey/response pattern
+  (survey0). Surveyor broadcasts a survey with a timed reply window
+  (`options.survey_time`, default 1s). Respondent echoes the backtrace
+  like REP. Shared `Routing::Backtrace` module extracted from REP.
+- **`NNQ::TimedOut`** error raised when the survey window expires.
+
 ## 0.4.0 — 2026-04-09
 
 - `Socket#all_peers_gone` — `Async::Promise` resolving the first time

data/lib/nnq/bus.rb

@@ -0,0 +1,37 @@
+# frozen_string_literal: true
+
+require_relative "socket"
+require_relative "routing/bus"
+
+module NNQ
+  # BUS (nng bus0): best-effort bidirectional mesh. Every message sent
+  # goes to all directly connected peers. Every message received from
+  # any peer is delivered to the application. Self-pairing (BUS ↔ BUS).
+  #
+  # Send never blocks — if a peer's queue is full, the message is
+  # dropped for that peer (matching nng's best-effort semantics).
+  #
+  class BUS0 < Socket
+    def send(body)
+      body = frozen_binary(body)
+      Reactor.run { @engine.routing.send(body) }
+    end
+
+
+    def receive
+      Reactor.run { @engine.routing.receive }
+    end
+
+
+    private
+
+    def protocol
+      Protocol::SP::Protocols::BUS_V0
+    end
+
+
+    def build_routing(engine)
+      Routing::Bus.new(engine)
+    end
+  end
+end

data/lib/nnq/connection.rb

@@ -12,9 +12,11 @@ module NNQ
     # @return [Protocol::SP::Connection]
     attr_reader :sp
 
+
     # @return [String, nil] endpoint URI we connected to / accepted from
     attr_reader :endpoint
 
+
     # @param sp [Protocol::SP::Connection] handshake-completed SP connection
     # @param endpoint [String, nil]
     def initialize(sp, endpoint: nil)
@@ -25,16 +27,20 @@ module NNQ
 
 
     # @return [Integer] peer protocol id (e.g. Protocols::PULL_V0)
-    def peer_protocol = @sp.peer_protocol
+    def peer_protocol
+      @sp.peer_protocol
+    end
 
 
     # Writes one message into the SP connection's send buffer (no flush).
     #
     # @param body [String]
+    # @param header [String, nil] optional binary prefix written between
+    #   the SP length prefix and body (see Protocol::SP::Connection)
     # @return [void]
-    def write_message(body)
+    def write_message(body, header: nil)
       raise ClosedError, "connection closed" if @closed
-      @sp.write_message(body)
+      @sp.write_message(body, header: header)
     end
 
 
@@ -53,10 +59,11 @@ module NNQ
     # each call is request-paced and there's nothing to batch.
     #
     # @param body [String]
+    # @param header [String, nil] optional binary prefix
     # @return [void]
-    def send_message(body)
+    def send_message(body, header: nil)
       raise ClosedError, "connection closed" if @closed
-      @sp.send_message(body)
+      @sp.send_message(body, header: header)
     end
 
 
@@ -77,7 +84,9 @@ module NNQ
 
 
     # @return [Boolean]
-    def closed? = @closed
+    def closed?
+      @closed
+    end
 
 
     # Closes the underlying SP connection. Safe to call twice.
@@ -86,5 +95,6 @@ module NNQ
       @closed = true
       @sp.close
     end
+
   end
 end

data/lib/nnq/engine/connection_lifecycle.rb

@@ -1,5 +1,6 @@
 # frozen_string_literal: true
 
+require "async/barrier"
 require "protocol/sp"
 require_relative "../connection"
 
@@ -42,6 +43,12 @@ module NNQ
       # @return [Symbol]
       attr_reader :state
 
+      # @return [Async::Barrier] holds all per-connection pump tasks
+      #   (send pump, recv pump). When the connection is torn down,
+      #   {#tear_down!} calls `@barrier.stop` to cancel every sibling
+      #   task atomically.
+      attr_reader :barrier
+
 
       # @param engine [Engine]
       # @param endpoint [String, nil]
@@ -52,6 +59,7 @@ module NNQ
         @framing  = framing
         @state    = :new
         @conn     = nil
+        @barrier  = Async::Barrier.new(parent: engine.barrier)
       end
 
 
@@ -68,13 +76,15 @@ module NNQ
           max_message_size: @engine.options.max_message_size,
           framing:          @framing,
         )
-        sp.handshake!
+        Async::Task.current.with_timeout(handshake_timeout) { sp.handshake! }
         ready!(NNQ::Connection.new(sp, endpoint: @endpoint))
         @conn
-      rescue => e
-        @engine.emit_monitor_event(:handshake_failed, endpoint: @endpoint, detail: { error: e })
+      rescue Protocol::SP::Error, *CONNECTION_LOST, Async::TimeoutError => error
+        @engine.emit_monitor_event(:handshake_failed, endpoint: @endpoint, detail: { error: error })
         io.close rescue nil
-        transition!(:closed) unless @state == :closed
+        # Full tear-down with reconnect: without this, the endpoint
+        # goes dead when a peer RSTs mid-handshake.
+        tear_down!(reconnect: true)
         raise
       end
 
@@ -83,16 +93,28 @@ module NNQ
       # asks the engine to schedule a reconnect (if the endpoint is in
       # the dialed set and reconnect is still enabled).
       def lost!
-        ep = @endpoint
-        tear_down!
-        @engine.maybe_reconnect(ep)
+        tear_down!(reconnect: true)
       end
 
 
       # Deliberate close (engine shutdown or routing eviction). Does
       # not trigger reconnect.
       def close!
-        tear_down!
+        tear_down!(reconnect: false)
+      end
+
+
+      # Starts a supervisor for this connection. Must be called after
+      # all per-connection pumps (recv loop, send pump) have been
+      # spawned on the connection barrier. The supervisor blocks until
+      # the first pump exits, then runs tear_down! via lost!.
+      #
+      # Called by Engine#handle_accepted / Engine#handle_connected after
+      # spawning the recv loop — routing's connection_added may have
+      # already spawned send pumps during ready!, so the barrier is
+      # guaranteed non-empty by then.
+      def start_supervisor!
+        start_supervisor unless @barrier.empty?
       end
 
 
@@ -106,7 +128,7 @@ module NNQ
           @engine.routing.connection_added(conn) if @engine.routing.respond_to?(:connection_added)
         rescue ConnectionRejected
           @engine.emit_monitor_event(:connection_rejected, endpoint: @endpoint)
-          tear_down!
+          tear_down!(reconnect: false)
           raise
         end
         @engine.lifecycle.peer_connected.resolve(conn) unless @engine.lifecycle.peer_connected.resolved?
@@ -116,7 +138,7 @@ module NNQ
       end
 
 
-      def tear_down!
+      def tear_down!(reconnect: false)
         return if @state == :closed
         transition!(:closed)
         if @conn
@@ -126,6 +148,35 @@ module NNQ
           @engine.emit_monitor_event(:disconnected, endpoint: @endpoint)
           @engine.resolve_all_peers_gone_if_empty
         end
+        @engine.maybe_reconnect(@endpoint) if reconnect
+        # Cancel every sibling pump of this connection. The caller is
+        # the supervisor task, which is NOT in the barrier — so there
+        # is no self-stop risk.
+        @barrier.stop
+      end
+
+
+      # Spawns a supervisor task on the *socket-level* barrier (not the
+      # per-connection barrier) that blocks on the first pump to finish
+      # and then triggers teardown.
+      def start_supervisor
+        @engine.barrier.async(transient: true, annotation: "conn supervisor") do
+          @barrier.wait { |task| task.wait; break }
+        rescue Async::Stop, Async::Cancel
+        rescue *CONNECTION_LOST
+        ensure
+          lost!
+        end
+      end
+
+
+      # Handshake timeout: same logic as TCP.connect_timeout — derived
+      # from reconnect_interval (floor 0.5s). Prevents a hang when the
+      # peer accepts the TCP connection but never sends an SP greeting.
+      def handshake_timeout
+        ri = @engine.options.reconnect_interval
+        ri = ri.end if ri.is_a?(Range)
+        [ri, 0.5].max
       end
 
 

data/lib/nnq/engine/reconnect.rb

@@ -55,10 +55,10 @@ module NNQ
       def run(parent_task, delay: nil)
         delay, max_delay = init_delay(delay)
 
-        task = parent_task.async(transient: true, annotation: "nnq reconnect #{@endpoint}") do
+        parent_task.async(transient: true, annotation: "nnq reconnect #{@endpoint}") do
           loop do
             break if @engine.closed?
-            sleep delay if delay > 0
+            sleep quantized_wait(delay) if delay > 0
             break if @engine.closed?
             begin
               @engine.transport_for(@endpoint).connect(@endpoint, @engine)
@@ -70,13 +70,22 @@ module NNQ
           end
         rescue Async::Stop
         end
-        @engine.tasks << task
       end
 
 
       private
 
 
+      # Wall-clock quantized sleep: wait until the next +delay+-sized
+      # grid tick. Multiple clients reconnecting with the same interval
+      # wake up at the same instant, collapsing staggered retries into
+      # aligned waves.
+      def quantized_wait(delay, now = Time.now.to_f)
+        wait = delay - (now % delay)
+        wait.positive? ? wait : delay
+      end
+
+
       def init_delay(delay)
         ri = @options.reconnect_interval
         if ri.is_a?(Range)

data/lib/nnq/engine/socket_lifecycle.rb

@@ -1,5 +1,6 @@
 # frozen_string_literal: true
 
+require "async/barrier"
 require "async/promise"
 
 module NNQ
@@ -42,9 +43,14 @@ module NNQ
       #   Edge-triggered: does not re-arm on reconnect.
       attr_reader :all_peers_gone
 
+      # @return [Async::Barrier, nil] holds every socket-scoped task
+      #   (connection supervisors, reconnect loops, accept loops).
+      #   {Engine#close} calls +barrier.stop+ to cascade teardown
+      #   through every per-connection barrier in one shot.
+      attr_reader :barrier
+
       # @return [Boolean] when false, the engine must not schedule new
-      #   reconnect attempts. Default true. nnq has no automatic
-      #   reconnect loop yet, so this currently just records intent.
+      #   reconnect attempts. Default true.
       attr_accessor :reconnect_enabled
 
 
@@ -55,6 +61,7 @@ module NNQ
         @peer_connected    = Async::Promise.new
         @all_peers_gone    = Async::Promise.new
         @reconnect_enabled = true
+        @barrier           = nil
       end
 
 
@@ -75,6 +82,7 @@ module NNQ
         return false if @parent_task
         @parent_task  = task
         @on_io_thread = on_io_thread
+        @barrier      = Async::Barrier.new(parent: @parent_task)
         transition!(:open)
         true
       end