nnq 0.6.0 → 0.7.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 0a336ac1e24bc6210ddeac6731163baab6f8980d9eebbe3235fac13157416fcf
-  data.tar.gz: f049bf9038235487966cae1c8920b452abf9984e3776b2b93cbbd95e067fb485
+  metadata.gz: c55340c77dffd3e4fdbc8419fb463e608a6e635adc577582e295b8af20fdc3e0
+  data.tar.gz: 6bb76f07c3732a4e69ad9a8e93a6a852f5ca04aac40fedc0df346ec9a2d74297
 SHA512:
-  metadata.gz: dcef45943a41f1bc53bbcf8ecc0c34c2779e4a16dea04e8f78854cf6a9debe11168596b47b764728e49393f61c319d41e2d79e489b37dec809adc59cbc0741f0
-  data.tar.gz: fd7aaa30c57d5b8fbfd6279aba8b96423e201837ca4ef0144a315ec020da5e0183aaede0c7327fc49bf1bd318894099e4fe02657a12aebf98c1d895582583c62
+  metadata.gz: 1cb190f96ace0a94363ea6a0b24b9d8b563363fbb3fb84e0bb4927a8ef818949270f20dec984828405bd15068400ace40aa5a9017d52321e2f9e10aa5235d973
+  data.tar.gz: 036a2a8203a7a26afad8d67fd23d9606d4947561ec0d8d31893b6efb116b62f623560afc6a4293fdbbd8846c39fa76aa9885eb47c9ab11900d552f5bc4377ecd

data/CHANGELOG.md

@@ -1,5 +1,94 @@
 # Changelog
 
+## 0.7.0 — 2026-04-18
+
+- **Inproc transport now uses a queue-based `Inproc::Pipe`** instead
+  of a Unix `socketpair(2)` running the full SP protocol.
+  `NNQ::Transport::Inproc::Pipe` duck-types `NNQ::Connection` and
+  transfers frozen Strings through a pair of `Async::Queue`s (one
+  per direction). No framing, no handshake, no kernel buffer copy.
+  When a routing strategy supplies an SP backtrace header
+  (REQ/REP/SURVEYOR), it's prepended before enqueue so the receive
+  side sees the same layout as the TCP/IPC path and `parse_backtrace`
+  keeps working unchanged. The new `Engine#connection_ready(conn,
+  endpoint:)` and `ConnectionLifecycle#ready_direct!` entry points
+  register a pipe as ready without the SP handshake phase.
+- **Inproc direct-recv fast path.** When a routing strategy exposes a
+  `#direct_recv_for(conn)` hook, the peer pipe enqueues directly into
+  the routing recv queue via `Pipe#wire_direct_recv`, bypassing both
+  the intermediate pipe queue and the recv pump fiber. PULL, BUS,
+  PAIR, SUB, REP, RESPONDENT, SURVEYOR, and the `*_raw` variants all
+  implement the hook; REQ (promise-based) stays on the fiber path.
+  Cuts three fiber hops to one on the steady-state recv path.
+
+  Inproc PUSH/PULL single-peer throughput (Ruby 4.0.2):
+
+  | Size   | Before (no JIT) | After (no JIT) | After (+YJIT) |
+  |---|---|---|---|
+  | 128 B  |  122k msg/s |  350k msg/s | 1,226k msg/s |
+  | 2 KiB  |   87k msg/s |  360k msg/s | 1,458k msg/s |
+  | 32 KiB |   21k msg/s |  261k msg/s |   887k msg/s |
+
+- **Routing pumps shed their `@pump_tasks` bookkeeping.** `bus`, `pub`,
+  `surveyor`, and `surveyor_raw` no longer track per-connection pump
+  tasks in a hash. Pumps are spawned under
+  `@engine.connections[conn].barrier`, so `ConnectionLifecycle#tear_down!`
+  already cascade-cancels them on `barrier.stop` — the hash was dead
+  weight.
+- **Transport registry is pluggable.** `NNQ::Engine.transports` is now a
+  mutable class-level `Hash` instead of a frozen constant; each built-in
+  transport (`tcp`, `ipc`, `inproc`) self-registers at load with
+  `Engine.transports["…"] = self`. External transports (e.g. `nnq-zstd`'s
+  `zstd+tcp://`) can register themselves the same way.
+- **`ConnectionLifecycle` calls `transport.wrap_connection(conn, engine)`
+  after handshake.** Transports that implement the hook can return a
+  delegating wrapper that layers compression / TLS / instrumentation
+  over the raw `NNQ::Connection` without the engine caring. Transports
+  without the hook (tcp/ipc/inproc) pass through unchanged.
+- **`lib/nnq.rb` restructured to mirror `lib/omq.rb`.** Requires split
+  into Core / Transport / Socket-types sections. New
+  `lib/nnq/constants.rb` owns `MonitorEvent`, the `CONNECTION_LOST` /
+  `CONNECTION_FAILED` error arrays, and `NNQ.freeze_for_ractors!` — all
+  previously scattered across `engine.rb`, `reconnect.rb`,
+  `monitor_event.rb`, and the top-level `nnq.rb`. `monitor_event.rb` is
+  removed (absorbed into constants).
+- **Benchmarks: richer scaffolding, measured via `Async::Clock`.**
+  `BenchHelper` gains `NNQ_BENCH_SIZES` / `NNQ_BENCH_TRANSPORTS` /
+  `NNQ_BENCH_PEERS` env overrides, a `measure_roundtrip` helper for
+  REQ/REP-style patterns, and a `wait_subscribed` helper that closes
+  the gap between TCP connect and SUBSCRIBE propagation. All elapsed
+  measurements use `Async::Clock.measure { … }` blocks instead of
+  `Process.clock_gettime`. `bench/report.rb --update-readme` now
+  falls back to the most recent row per cell across all history, so a
+  partial bench run refreshes only the cells it covers instead of
+  clobbering untouched cells with "—".
+
+## 0.6.1 — 2026-04-15
+
+- **Verbose trace (`-vvv`) now fires for cooked REQ/REP/RESPONDENT
+  sends.** Cooked `Req#send_request`, `Rep#send_reply`, and
+  `Respondent#send_reply` bypass `send_pump` and write to the
+  connection directly, so they were never emitting `:message_sent`
+  monitor events — `-vvv` only ever showed the `<<` recv side. Each
+  now calls `emit_verbose_msg_sent(body)` after the write. Raw
+  REQ/REP/RESPONDENT sends get the same treatment (raw surveyor
+  already emitted via its per-peer send pump).
+- **Verbose recv previews strip the SP backtrace header.** The recv
+  loop used to emit the raw wire body, so `-vvv` traces for
+  REQ/REP/SURVEYOR/RESPONDENT showed the 4-byte request/survey id
+  (or a multi-word backtrace stack) in front of the payload. Routing
+  strategies now expose an optional `preview_body(wire)` hook; the
+  engine calls it before emitting `:message_received` so the trace
+  shows just the payload.
+- **`Engine#close` drains the monitor queue before cancelling
+  tasks.** The monitor consumer fiber lives under the socket-level
+  barrier, so `barrier.stop` used to `Async::Stop` it before it had
+  a chance to drain trailing events. `close` now emits `:closed`,
+  enqueues the nil sentinel, and awaits the stored `monitor_task`
+  before stopping the barrier. Fixes flaky `-vvv` traces on
+  short-lived sockets where the last `:message_received` event
+  would occasionally be lost.
+
 ## 0.6.0 — 2026-04-15
 
 - **NNG-style raw mode for REQ/REP and SURVEYOR/RESPONDENT.** Constructing

data/lib/nnq/constants.rb

@@ -0,0 +1,58 @@
+# frozen_string_literal: true
+
+require "socket"
+require "io/stream"
+
+module NNQ
+  # Lifecycle event emitted by {Socket#monitor}.
+  #
+  # @!attribute [r] type
+  #   @return [Symbol] event type (:listening, :connected, :disconnected, ...)
+  # @!attribute [r] endpoint
+  #   @return [String, nil] the endpoint involved
+  # @!attribute [r] detail
+  #   @return [Hash, nil] extra context
+  #
+  MonitorEvent = Data.define(:type, :endpoint, :detail) do
+    def initialize(type:, endpoint: nil, detail: nil)
+      super
+    end
+  end
+
+
+  # Errors that indicate an established connection went away. Used by
+  # the recv loop, routing pumps, and connection lifecycle to silently
+  # terminate (the connection lifecycle's #lost! handler decides
+  # whether to reconnect). Not frozen at load time — transport plugins
+  # append to this before the first bind/connect, which freezes both
+  # arrays.
+  CONNECTION_LOST = [
+    EOFError,
+    IOError,
+    Errno::ECONNRESET,
+    Errno::EPIPE,
+  ]
+
+
+  # Errors raised when a peer cannot be reached. Triggers a reconnect
+  # retry rather than propagating.
+  CONNECTION_FAILED = [
+    Errno::ECONNREFUSED,
+    Errno::EHOSTUNREACH,
+    Errno::ENETUNREACH,
+    Errno::ENOENT,
+    Errno::EPIPE,
+    Errno::ETIMEDOUT,
+    Socket::ResolutionError,
+  ]
+
+
+  # Freezes module-level state so NNQ sockets can be used inside Ractors.
+  # Call this once before spawning any Ractors that create NNQ sockets.
+  #
+  def self.freeze_for_ractors!
+    CONNECTION_LOST.freeze
+    CONNECTION_FAILED.freeze
+    Engine.transports.freeze
+  end
+end

data/lib/nnq/engine/connection_lifecycle.rb

@@ -104,6 +104,15 @@ module NNQ
       end
 
 
+      # Registers an already-ready connection object (e.g. an
+      # {Transport::Inproc::Pipe}) without running the SP handshake.
+      # Transports that frame on the wire should call {#handshake!}
+      # instead.
+      def ready_direct!(conn)
+        ready!(conn)
+      end
+
+
       # Starts a supervisor for this connection. Must be called after
       # all per-connection pumps (recv loop, send pump) have been
       # spawned on the connection barrier. The supervisor blocks until
@@ -121,6 +130,7 @@ module NNQ
       private
 
       def ready!(conn)
+        conn                     = wrap_connection(conn)
         @conn                    = conn
         @engine.connections[conn] = self
         transition!(:ready)
@@ -170,6 +180,19 @@ module NNQ
       end
 
 
+      # Post-handshake transport wrap. A transport that implements
+      # `wrap_connection(conn)` (e.g. nnq-zstd's zstd+tcp) returns a
+      # delegating wrapper that adds a layer (compression, TLS, …)
+      # without the engine caring. Unknown or hook-less transports pass
+      # through unchanged.
+      def wrap_connection(conn)
+        return conn unless @endpoint
+        transport = @engine.transport_for(@endpoint)
+        return conn unless transport.respond_to?(:wrap_connection)
+        transport.wrap_connection(conn, @engine)
+      end
+
+
       # Handshake timeout: same logic as TCP.connect_timeout — derived
       # from reconnect_interval (floor 0.5s). Prevents a hang when the
       # peer accepts the TCP connection but never sends an SP greeting.

data/lib/nnq/engine/reconnect.rb

@@ -2,31 +2,6 @@
 
 module NNQ
   class Engine
-    # Connection errors that should trigger a reconnect retry rather
-    # than propagate. Mutable at load time so plugins (e.g. a future
-    # TLS transport) can append their own error classes; frozen on
-    # first {Engine#connect}.
-    CONNECTION_FAILED = [
-      Errno::ECONNREFUSED,
-      Errno::EHOSTUNREACH,
-      Errno::ENETUNREACH,
-      Errno::ENOENT,
-      Errno::EPIPE,
-      Errno::ETIMEDOUT,
-      Socket::ResolutionError,
-    ]
-
-    # Errors that indicate an established connection went away. Used
-    # by the recv loop and pumps to silently terminate (the connection
-    # lifecycle's #lost! handler decides whether to reconnect).
-    CONNECTION_LOST = [
-      EOFError,
-      IOError,
-      Errno::ECONNRESET,
-      Errno::EPIPE,
-    ]
-
-
     # Schedules reconnect attempts with exponential back-off.
     #
     # Runs a background task that loops until a connection is
@@ -61,7 +36,7 @@ module NNQ
             sleep quantized_wait(delay) if delay > 0
             break if @engine.closed?
             begin
-              @engine.transport_for(@endpoint).connect(@endpoint, @engine)
+              @engine.transport_for(@endpoint).connect(@endpoint, @engine, **@engine.dial_opts_for(@endpoint))
               break
             rescue *CONNECTION_FAILED, *CONNECTION_LOST => e
               delay = next_delay(delay, max_delay)

data/lib/nnq/engine.rb

@@ -4,16 +4,9 @@ require "async"
 require "async/clock"
 require "set"
 require "protocol/sp"
-require_relative "error"
-require_relative "connection"
-require_relative "monitor_event"
-require_relative "reactor"
 require_relative "engine/socket_lifecycle"
 require_relative "engine/connection_lifecycle"
 require_relative "engine/reconnect"
-require_relative "transport/tcp"
-require_relative "transport/ipc"
-require_relative "transport/inproc"
 
 module NNQ
   # Per-socket orchestrator. Owns the listener set, the connection map
@@ -25,11 +18,18 @@ module NNQ
   # no HWM bookkeeping, no mechanisms, no heartbeat, no monitor queue.
   #
   class Engine
-    TRANSPORTS = {
-      "tcp"    => Transport::TCP,
-      "ipc"    => Transport::IPC,
-      "inproc" => Transport::Inproc,
-    }
+    # Scheme → transport module registry. Each transport file
+    # self-registers on require; plugins (e.g. nnq-zstd) add more:
+    #
+    #   NNQ::Engine.transports["zstd+tcp"] = NNQ::Transport::ZstdTcp
+    #
+    @transports = {}
+
+
+    class << self
+      # @return [Hash{String => Module}] registered transports
+      attr_reader :transports
+    end
 
 
     # @return [Integer] our SP protocol id (e.g. Protocols::PUSH_V0)
@@ -70,6 +70,10 @@ module NNQ
     attr_accessor :monitor_queue
 
 
+    # @return [Async::Task, nil] the monitor consumer task, if any
+    attr_accessor :monitor_task
+
+
     # @return [Boolean] when true, {#emit_verbose_monitor_event} forwards
     #   per-message traces (:message_sent / :message_received) to the
     #   monitor queue. Set by {Socket#monitor} via its +verbose:+ kwarg.
@@ -91,6 +95,7 @@ module NNQ
       @monitor_queue   = nil
       @verbose_monitor = false
       @dialed          = Set.new
+      @dial_opts       = {} # endpoint => kwargs for transport.connect on reconnect
       @routing         = yield(self)
     end
 
@@ -187,9 +192,9 @@ module NNQ
 
 
     # Binds to +endpoint+. Synchronous: errors propagate.
-    def bind(endpoint)
+    def bind(endpoint, **opts)
       transport = transport_for(endpoint)
-      listener  = transport.bind(endpoint, self)
+      listener  = transport.bind(endpoint, self, **opts)
       listener.start_accept_loop(@lifecycle.barrier) do |io, framing = :tcp|
         handle_accepted(io, endpoint: endpoint, framing: framing)
       end
@@ -203,12 +208,13 @@ module NNQ
     # actual dial happens inside a background reconnect task that
     # retries with exponential back-off until the peer becomes
     # reachable. Inproc connect is synchronous and instant.
-    def connect(endpoint)
+    def connect(endpoint, **opts)
       @dialed << endpoint
+      @dial_opts[endpoint] = opts unless opts.empty?
       @last_endpoint = endpoint
 
       if endpoint.start_with?("inproc://")
-        transport_for(endpoint).connect(endpoint, self)
+        transport_for(endpoint).connect(endpoint, self, **opts)
       else
         emit_monitor_event(:connect_delayed, endpoint: endpoint)
         Reconnect.schedule(endpoint, @options, @lifecycle.barrier, self, delay: 0)
@@ -216,6 +222,14 @@ module NNQ
     end
 
 
+    # Transport options captured from {#connect} for +endpoint+. Used by
+    # {Reconnect} to re-dial with the original kwargs. Empty hash for
+    # endpoints connected without extra options.
+    def dial_opts_for(endpoint)
+      @dial_opts[endpoint] || {}
+    end
+
+
     # Schedules a reconnect for +endpoint+ if auto-reconnect is enabled
     # and the endpoint is still in the dialed set. Called from the
     # connection lifecycle's `lost!` path.
@@ -231,7 +245,7 @@ module NNQ
     # transport from the URL each iteration.
     def transport_for(endpoint)
       scheme = endpoint[/\A([a-z+]+):\/\//i, 1] or raise Error, "no scheme: #{endpoint}"
-      TRANSPORTS[scheme] or raise Error, "unsupported transport: #{scheme}"
+      Engine.transports[scheme] or raise Error, "unsupported transport: #{scheme}"
     end
 
 
@@ -259,6 +273,19 @@ module NNQ
     end
 
 
+    # Registers an already-connected, framing-free pipe (inproc). Skips
+    # the SP handshake entirely — {Transport::Inproc::Pipe} is a Ruby
+    # duck-type for {NNQ::Connection} and has no wire protocol.
+    def connection_ready(conn, endpoint:)
+      lifecycle = ConnectionLifecycle.new(self, endpoint: endpoint, framing: :inproc)
+      lifecycle.ready_direct!(conn)
+      spawn_recv_loop(conn) if @routing.respond_to?(:enqueue) && @connections.key?(conn)
+      lifecycle.start_supervisor!
+    rescue ConnectionRejected
+      # routing rejected this peer (e.g. PAIR already bonded)
+    end
+
+
     # Spawns a task under the given parent barrier (defaults to the
     # socket-level barrier). Used by routing strategies (e.g. PUSH send
     # pump) to attach long-lived fibers to the engine's lifecycle. The
@@ -286,6 +313,15 @@ module NNQ
       # collection mutates during iteration, so snapshot the values.
       @connections.values.each(&:close!)
 
+      # Emit :closed, seal the monitor queue, and wait for the monitor
+      # fiber to drain it before cancelling tasks. Without this join,
+      # trailing :message_received events that the recv pump enqueued
+      # just before close would be lost when the barrier.stop below
+      # Async::Stops the monitor fiber mid-dequeue.
+      emit_monitor_event(:closed)
+      close_monitor_queue
+      @monitor_task&.wait
+
       # Cascade-cancel every remaining task (reconnect loops, accept
       # loops, supervisors) in one shot.
       @lifecycle.barrier&.stop
@@ -295,8 +331,6 @@ module NNQ
       # Unblock anyone waiting on peer_connected when the socket is
       # closed before a peer ever arrived.
       @lifecycle.peer_connected.resolve(nil) unless @lifecycle.peer_connected.resolved?
-      emit_monitor_event(:closed)
-      close_monitor_queue
     end
 
 
@@ -334,10 +368,25 @@ module NNQ
 
 
     def spawn_recv_loop(conn)
+      # Inproc fast-path: wire the peer pipe to enqueue directly into
+      # the routing recv queue, skipping both the recv pump fiber and
+      # the intermediate pipe queue. Cuts three fiber hops to one on
+      # PUSH/PULL and peers.
+      if conn.is_a?(Transport::Inproc::Pipe) && conn.peer && @routing.respond_to?(:direct_recv_for)
+        queue, transform = @routing.direct_recv_for(conn)
+        if queue
+          conn.peer.wire_direct_recv(queue, transform)
+          return
+        end
+      end
+
       @connections[conn].barrier.async(annotation: "nnq recv #{conn.endpoint}") do
         loop do
           body = conn.receive_message
-          emit_verbose_msg_received(body)
+          if @verbose_monitor
+            preview = @routing.respond_to?(:preview_body) ? @routing.preview_body(body) : body
+            emit_verbose_msg_received(preview)
+          end
           @routing.enqueue(body, conn)
         rescue *CONNECTION_LOST, Async::Stop
           break

data/lib/nnq/routing/bus.rb

@@ -22,7 +22,6 @@ module NNQ
       def initialize(engine)
         @engine     = engine
         @queues     = {} # conn => Async::LimitedQueue
-        @pump_tasks = {} # conn => Async::Task
         @recv_queue = Async::Queue.new
       end
 
@@ -44,6 +43,13 @@ module NNQ
       end
 
 
+      # Inproc fast-path hook: peer pipe enqueues directly into the
+      # shared recv queue — identity transform, no backtrace or filter.
+      def direct_recv_for(_conn)
+        [@recv_queue, nil]
+      end
+
+
       # @return [String, nil] message body, or nil once the socket is closed
       def receive
         @recv_queue.dequeue
@@ -52,18 +58,13 @@ module NNQ
 
       def connection_added(conn)
         queue = Async::LimitedQueue.new(@engine.options.send_hwm)
-        @queues[conn]     = queue
-        @pump_tasks[conn] = spawn_pump(conn, queue)
+        @queues[conn] = queue
+        spawn_pump(conn, queue)
       end
 
 
       def connection_removed(conn)
         @queues.delete(conn)
-        task = @pump_tasks.delete(conn)
-        return unless task
-        return if task == Async::Task.current
-        task.stop
-      rescue IOError, Errno::EPIPE
       end
 
 
@@ -73,8 +74,6 @@ module NNQ
 
 
       def close
-        @pump_tasks.each_value(&:stop)
-        @pump_tasks.clear
         @queues.clear
         @recv_queue.enqueue(nil)
       end

data/lib/nnq/routing/pair.rb

@@ -48,6 +48,13 @@ module NNQ
       end
 
 
+      # Inproc fast-path hook: peer pipe enqueues straight into the
+      # local recv queue.
+      def direct_recv_for(_conn)
+        [@recv_queue, nil]
+      end
+
+
       # First-pipe-wins. Raising {ConnectionRejected} tells the
       # ConnectionLifecycle to tear down the just-registered connection
       # without ever exposing it to pumps.

data/lib/nnq/routing/pub.rb

@@ -19,9 +19,8 @@ module NNQ
     #
     class Pub
       def initialize(engine)
-        @engine     = engine
-        @queues     = {} # conn => Async::LimitedQueue
-        @pump_tasks = {} # conn => Async::Task
+        @engine = engine
+        @queues = {} # conn => Async::LimitedQueue
       end
 
 
@@ -42,19 +41,13 @@ module NNQ
         # control into the new task body, which parks on queue.dequeue;
         # at that park the publisher fiber can run and must already see
         # this peer's queue.
-        @queues[conn]     = queue
-        @pump_tasks[conn] = spawn_pump(conn, queue)
+        @queues[conn] = queue
+        spawn_pump(conn, queue)
       end
 
 
       def connection_removed(conn)
         @queues.delete(conn)
-        task = @pump_tasks.delete(conn)
-        return unless task
-        return if task == Async::Task.current
-        task.stop
-      rescue IOError, Errno::EPIPE
-        # pump was mid-flush; already unwinding
       end
 
 
@@ -65,8 +58,6 @@ module NNQ
 
 
       def close
-        @pump_tasks.each_value(&:stop)
-        @pump_tasks.clear
         @queues.clear
       end
 

data/lib/nnq/routing/pull.rb

@@ -22,6 +22,14 @@ module NNQ
       end
 
 
+      # Inproc fast-path hook: return the routing recv queue so the
+      # peer pipe can enqueue directly, skipping the recv pump fiber.
+      # Identity transform — PULL bodies are the user payload already.
+      def direct_recv_for(_conn)
+        [@queue, nil]
+      end
+
+
       # @return [String, nil] message body, or nil if the queue was closed
       def receive
         @queue.dequeue

data/lib/nnq/routing/rep.rb

@@ -70,6 +70,14 @@ module NNQ
 
         return if conn.closed?
         conn.send_message(body, header: btrace)
+        @engine.emit_verbose_msg_sent(body)
+      end
+
+
+      # Strips the backtrace header for verbose trace previews.
+      def preview_body(wire)
+        _, payload = parse_backtrace(wire)
+        payload || wire
       end
 
 
@@ -81,6 +89,17 @@ module NNQ
       end
 
 
+      # Inproc fast-path hook: peer pipe parses the backtrace and
+      # enqueues the same [conn, btrace, payload] tuple the pump would.
+      def direct_recv_for(conn)
+        transform = lambda do |body|
+          btrace, payload = parse_backtrace(body)
+          btrace ? [conn, btrace, payload] : nil
+        end
+        [@recv_queue, transform]
+      end
+
+
       def connection_removed(conn)
         @mutex.synchronize do
           @pending = nil if @pending && @pending[0] == conn

data/lib/nnq/routing/rep_raw.rb

@@ -36,11 +36,18 @@ module NNQ
         return if to.closed?
         return if Backtrace.too_many_hops?(header)
         to.send_message(body, header: header)
+        @engine.emit_verbose_msg_sent(body)
       rescue ClosedError
         # peer went away between receive and send — drop
       end
 
 
+      def preview_body(wire)
+        _, payload = parse_backtrace(wire)
+        payload || wire
+      end
+
+
       # Called by the engine recv loop.
       def enqueue(wire_bytes, conn)
         header, payload = parse_backtrace(wire_bytes)
@@ -49,6 +56,16 @@ module NNQ
       end
 
 
+      # Inproc fast-path hook.
+      def direct_recv_for(conn)
+        transform = lambda do |wire_bytes|
+          header, payload = parse_backtrace(wire_bytes)
+          header ? [conn, header, payload] : nil
+        end
+        [@recv_queue, transform]
+      end
+
+
       def close
         @recv_queue.enqueue(nil)
       end

data/lib/nnq/routing/req.rb

@@ -56,6 +56,7 @@ module NNQ
         conn   = pick_peer
         header = [id].pack("N")
         conn.send_message(body, header: header)
+        @engine.emit_verbose_msg_sent(body)
         promise.wait
       ensure
         @mutex.synchronize do
@@ -81,6 +82,12 @@ module NNQ
       end
 
 
+      # Strips the 4-byte request id for verbose trace previews.
+      def preview_body(wire)
+        wire.byteslice(4..) || wire
+      end
+
+
       def close
         @mutex.synchronize do
           @outstanding&.last&.reject(NNQ::Error.new("REQ socket closed"))

data/lib/nnq/routing/req_raw.rb

@@ -26,6 +26,13 @@ module NNQ
       def send(body, header:)
         conn = pick_peer
         conn.send_message(body, header: header)
+        @engine.emit_verbose_msg_sent(body)
+      end
+
+
+      def preview_body(wire)
+        _, payload = parse_backtrace(wire)
+        payload || wire
       end
 
 
@@ -41,6 +48,16 @@ module NNQ
       end
 
 
+      # Inproc fast-path hook.
+      def direct_recv_for(conn)
+        transform = lambda do |wire_bytes|
+          header, payload = parse_backtrace(wire_bytes)
+          header ? [conn, header, payload] : nil
+        end
+        [@recv_queue, transform]
+      end
+
+
       def close
         @recv_queue.enqueue(nil)
       end

data/lib/nnq/routing/respondent.rb

@@ -52,6 +52,14 @@ module NNQ
 
         return if conn.closed?
         conn.send_message(body, header: btrace)
+        @engine.emit_verbose_msg_sent(body)
+      end
+
+
+      # Strips the backtrace header for verbose trace previews.
+      def preview_body(wire)
+        _, payload = parse_backtrace(wire)
+        payload || wire
       end
 
 
@@ -63,6 +71,16 @@ module NNQ
       end
 
 
+      # Inproc fast-path hook.
+      def direct_recv_for(conn)
+        transform = lambda do |body|
+          btrace, payload = parse_backtrace(body)
+          btrace ? [conn, btrace, payload] : nil
+        end
+        [@recv_queue, transform]
+      end
+
+
       def connection_removed(conn)
         @mutex.synchronize do
           @pending = nil if @pending && @pending[0] == conn

data/lib/nnq/routing/respondent_raw.rb

@@ -29,10 +29,17 @@ module NNQ
         return if to.closed?
         return if Backtrace.too_many_hops?(header)
         to.send_message(body, header: header)
+        @engine.emit_verbose_msg_sent(body)
       rescue ClosedError
       end
 
 
+      def preview_body(wire)
+        _, payload = parse_backtrace(wire)
+        payload || wire
+      end
+
+
       def enqueue(wire_bytes, conn)
         header, payload = parse_backtrace(wire_bytes)
         return unless header
@@ -40,6 +47,16 @@ module NNQ
       end
 
 
+      # Inproc fast-path hook.
+      def direct_recv_for(conn)
+        transform = lambda do |wire_bytes|
+          header, payload = parse_backtrace(wire_bytes)
+          header ? [conn, header, payload] : nil
+        end
+        [@recv_queue, transform]
+      end
+
+
       def close
         @recv_queue.enqueue(nil)
       end

data/lib/nnq/routing/sub.rb

@@ -38,6 +38,13 @@ module NNQ
       end
 
 
+      # Inproc fast-path hook: filter via the subscription list in the
+      # transform, then enqueue only matching bodies.
+      def direct_recv_for(_conn)
+        [@queue, ->(body) { matches?(body) ? body : nil }]
+      end
+
+
       # @return [String, nil]
       def receive
         @queue.dequeue

data/lib/nnq/routing/surveyor.rb

@@ -24,7 +24,6 @@ module NNQ
       def initialize(engine)
         @engine     = engine
         @queues     = {} # conn => Async::LimitedQueue
-        @pump_tasks = {} # conn => Async::Task
         @recv_queue = Async::Queue.new
         @current_id = nil
         @mutex      = Mutex.new
@@ -78,22 +77,36 @@ module NNQ
       end
 
 
+      # Inproc fast-path hook. Transform filters replies by current
+      # survey id and strips the 4-byte header, mirroring #enqueue.
+      def direct_recv_for(_conn)
+        mutex = @mutex
+        transform = lambda do |body|
+          next nil if body.bytesize < 4
+          id      = body.unpack1("N")
+          payload = body.byteslice(4..)
+          match   = mutex.synchronize { @current_id == id }
+          match ? payload : nil
+        end
+        [@recv_queue, transform]
+      end
+
+
+      # Strips the 4-byte survey id for verbose trace previews.
+      def preview_body(wire)
+        wire.byteslice(4..) || wire
+      end
+
+
       def connection_added(conn)
-        queue             = Async::LimitedQueue.new(@engine.options.send_hwm)
-        @queues[conn]     = queue
-        @pump_tasks[conn] = spawn_pump(conn, queue)
+        queue         = Async::LimitedQueue.new(@engine.options.send_hwm)
+        @queues[conn] = queue
+        spawn_pump(conn, queue)
       end
 
 
       def connection_removed(conn)
         @queues.delete(conn)
-        task = @pump_tasks.delete(conn)
-
-        return unless task
-        return if task == Async::Task.current
-
-        task.stop
-      rescue IOError, Errno::EPIPE
       end
 
 
@@ -103,8 +116,6 @@ module NNQ
 
 
       def close
-        @pump_tasks.each_value(&:stop)
-        @pump_tasks.clear
         @queues.clear
         @recv_queue.enqueue(nil)
       end

data/lib/nnq/routing/surveyor_raw.rb

@@ -23,7 +23,6 @@ module NNQ
       def initialize(engine)
         @engine     = engine
         @queues     = {} # conn => Async::LimitedQueue
-        @pump_tasks = {} # conn => Async::Task
         @recv_queue = Async::LimitedQueue.new(engine.options.recv_hwm)
       end
 
@@ -47,22 +46,31 @@ module NNQ
       end
 
 
+      # Inproc fast-path hook.
+      def direct_recv_for(conn)
+        transform = lambda do |wire_bytes|
+          header, payload = parse_backtrace(wire_bytes)
+          header ? [conn, header, payload] : nil
+        end
+        [@recv_queue, transform]
+      end
+
+
+      def preview_body(wire)
+        _, payload = parse_backtrace(wire)
+        payload || wire
+      end
+
+
       def connection_added(conn)
-        queue             = Async::LimitedQueue.new(@engine.options.send_hwm)
-        @queues[conn]     = queue
-        @pump_tasks[conn] = spawn_pump(conn, queue)
+        queue         = Async::LimitedQueue.new(@engine.options.send_hwm)
+        @queues[conn] = queue
+        spawn_pump(conn, queue)
       end
 
 
       def connection_removed(conn)
         @queues.delete(conn)
-        task = @pump_tasks.delete(conn)
-
-        return unless task
-        return if task == Async::Task.current
-
-        task.stop
-      rescue IOError, Errno::EPIPE
       end
 
 
@@ -72,8 +80,6 @@ module NNQ
 
 
       def close
-        @pump_tasks.each_value(&:stop)
-        @pump_tasks.clear
         @queues.clear
         @recv_queue.enqueue(nil)
       end

data/lib/nnq/socket.rb

@@ -2,11 +2,6 @@
 
 require "async/queue"
 
-require_relative "options"
-require_relative "engine"
-require_relative "monitor_event"
-require_relative "reactor"
-
 module NNQ
   # Socket base class. Subclasses (PUSH, PULL, ...) wire up a routing
   # strategy and the SP protocol id.
@@ -50,15 +45,15 @@ module NNQ
     end
 
 
-    def bind(endpoint)
+    def bind(endpoint, **opts)
       ensure_parent_task
-      Reactor.run { @engine.bind(endpoint) }
+      Reactor.run { @engine.bind(endpoint, **opts) }
     end
 
 
-    def connect(endpoint)
+    def connect(endpoint, **opts)
       ensure_parent_task
-      Reactor.run { @engine.connect(endpoint) }
+      Reactor.run { @engine.connect(endpoint, **opts) }
     end
 
 
@@ -126,13 +121,14 @@ module NNQ
       @engine.verbose_monitor = verbose
 
       Reactor.run do
-        @engine.spawn_task(annotation: "nnq monitor") do
+        @engine.monitor_task = @engine.spawn_task(annotation: "nnq monitor") do
           while (event = queue.dequeue)
             block.call(event)
           end
         rescue Async::Stop
         ensure
           @engine.monitor_queue = nil
+          @engine.monitor_task  = nil
           block.call(MonitorEvent.new(type: :monitor_stopped))
         end
       end

data/lib/nnq/transport/inproc/pipe.rb

@@ -0,0 +1,131 @@
+# frozen_string_literal: true
+
+module NNQ
+  module Transport
+    module Inproc
+      # Queue-based in-process pipe. Duck-types {NNQ::Connection} so
+      # routing strategies, the recv loop, and the send pump work
+      # against it unchanged.
+      #
+      # No wire framing: bodies are transferred as frozen Strings
+      # through a pair of {Async::Queue} (one per direction). When an
+      # SP backtrace header is supplied (REQ/REP/SURVEYOR paths), it's
+      # prepended before enqueue so {#receive_message} returns an
+      # already-prefixed body — matching the TCP/IPC framing semantic
+      # so routing's `parse_backtrace` parses the same layout either
+      # way.
+      #
+      # Direct-recv fast path: when a routing strategy calls
+      # {#wire_direct_recv} on the peer side of a pipe pair, subsequent
+      # {#send_message} calls enqueue straight into the consumer's
+      # recv queue — the intermediate pipe queue and the recv pump
+      # fiber are both skipped. Cuts three fiber hops to one and is
+      # what lets inproc PUSH/PULL clear 1M msg/s on YJIT.
+      #
+      # Wiring happens synchronously inside {Transport::Inproc.connect}
+      # (before the call returns to the caller), so there's no window
+      # in which a send can precede a wire — no pending buffer needed.
+      #
+      # Close protocol: {#close} enqueues a `nil` sentinel onto the
+      # send side (or the direct queue if wired). The peer's recv loop
+      # sees `nil`, raises `EOFError`, and unwinds via its connection
+      # supervisor.
+      class Pipe
+        # @return [String, nil] endpoint URI this pipe was established on
+        attr_reader :endpoint
+
+        # @return [Pipe, nil] the other end of the pair
+        attr_accessor :peer
+
+        # @return [Async::Queue, nil] when non-nil, {#send_message}
+        #   enqueues here instead of into @send_queue.
+        attr_reader :direct_recv_queue
+
+
+        def initialize(send_queue:, recv_queue:, endpoint:)
+          @send_queue            = send_queue
+          @recv_queue            = recv_queue
+          @endpoint              = endpoint
+          @closed                = false
+          @peer                  = nil
+          @direct_recv_queue     = nil
+          @direct_recv_transform = nil
+        end
+
+
+        # Wires the direct-recv fast path. After this call, messages
+        # sent on this pipe bypass the intermediate pipe queue and
+        # land directly in +queue+.
+        #
+        # @param queue [Async::Queue]
+        # @param transform [Proc, nil] optional per-message transform;
+        #   return nil to drop the message (used by filter/parse
+        #   strategies like SUB or REP).
+        def wire_direct_recv(queue, transform)
+          @direct_recv_transform = transform
+          @direct_recv_queue     = queue
+        end
+
+
+        def send_message(body, header: nil)
+          raise ClosedError, "connection closed" if @closed
+          wire = header ? header + body : body
+
+          if (q = @direct_recv_queue)
+            item = @direct_recv_transform ? @direct_recv_transform.call(wire) : wire
+            q.enqueue(item) unless item.nil?
+          else
+            @send_queue.enqueue(wire)
+          end
+        end
+
+
+        alias write_message send_message
+
+
+        def write_messages(bodies)
+          raise ClosedError, "connection closed" if @closed
+
+          if (q = @direct_recv_queue)
+            transform = @direct_recv_transform
+            bodies.each do |body|
+              item = transform ? transform.call(body) : body
+              q.enqueue(item) unless item.nil?
+            end
+          else
+            bodies.each { |body| @send_queue.enqueue(body) }
+          end
+        end
+
+
+        # No-op — Async::Queue has no IO buffer to flush.
+        def flush
+          nil
+        end
+
+
+        def receive_message
+          item = @recv_queue.dequeue
+          raise EOFError, "connection closed" if item.nil?
+          item
+        end
+
+
+        def closed?
+          @closed
+        end
+
+
+        def close
+          return if @closed
+          @closed = true
+          # Close sentinel goes on whichever queue the peer is reading.
+          # When direct-wired, @send_queue is unused; hit the direct
+          # queue so the consumer unblocks.
+          (@direct_recv_queue || @send_queue).enqueue(nil)
+        end
+
+      end
+    end
+  end
+end

data/lib/nnq/transport/inproc.rb

@@ -1,21 +1,24 @@
 # frozen_string_literal: true
 
-require "socket"
-require "io/stream"
+require "async/queue"
+
+require_relative "inproc/pipe"
 
 module NNQ
   module Transport
     # In-process transport. Both peers live in the same process and
-    # exchange frames over a Unix socketpair — no network, no address.
+    # exchange frozen Strings through a pair of {Async::Queue}s — no
+    # wire framing, no socketpair, no SP handshake.
     #
-    # Unlike omq's DirectPipe, inproc here still runs through
-    # Protocol::SP: the socketpair just replaces TCP. Kernel buffering
-    # across the pair is plenty to avoid contention for typical
-    # in-process message sizes, and reusing the SP handshake + framing
-    # keeps the transport ~40 LOC instead of a parallel Connection
-    # implementation.
+    # The historical implementation ran through a Unix `socketpair(2)`
+    # and the full SP protocol, making inproc roughly as expensive as
+    # IPC. Swapping to {Inproc::Pipe} (duck-types {NNQ::Connection})
+    # drops the kernel buffer copy, the framing encode/decode, and the
+    # handshake — inproc becomes a pure in-process queue transfer.
     #
     module Inproc
+      Engine.transports["inproc"] = self
+
       @registry = {}
       @mutex    = Mutex.new
 
@@ -26,7 +29,7 @@ module NNQ
         # @param endpoint [String] e.g. "inproc://my-endpoint"
         # @param engine [Engine]
         # @return [Listener]
-        def bind(endpoint, engine)
+        def bind(endpoint, engine, **)
           @mutex.synchronize do
             raise Error, "inproc endpoint already bound: #{endpoint}" if @registry.key?(endpoint)
             @registry[endpoint] = engine
@@ -36,28 +39,27 @@ module NNQ
         end
 
 
-        # Connects +engine+ to a bound inproc endpoint. Creates a Unix
-        # socketpair, hands one side to the bound engine (accepted),
-        # the other to the connecting engine (connected). Both sides
-        # run the normal SP handshake concurrently.
+        # Connects +engine+ to a bound inproc endpoint. Creates a Pipe
+        # pair — one queue per direction — and registers each side with
+        # its owning engine via {Engine#connection_ready}. No handshake
+        # runs; both ends are live as soon as the pipes are wired.
         #
         # @param endpoint [String]
         # @param engine [Engine]
         # @return [void]
-        def connect(endpoint, engine)
+        def connect(endpoint, engine, **)
           bound = @mutex.synchronize { @registry[endpoint] }
           raise Error, "inproc endpoint not bound: #{endpoint}" unless bound
 
-          a, b = UNIXSocket.pair
+          a_to_b = Async::Queue.new
+          b_to_a = Async::Queue.new
+          client = Pipe.new(send_queue: a_to_b, recv_queue: b_to_a, endpoint: endpoint)
+          server = Pipe.new(send_queue: b_to_a, recv_queue: a_to_b, endpoint: endpoint)
+          client.peer = server
+          server.peer = client
 
-          # Handshake on the bound side must run concurrently with
-          # ours — if we called bound.handle_accepted synchronously
-          # it would block on reading our greeting before we've had
-          # a chance to write it.
-          bound.spawn_task(annotation: "nnq inproc accept #{endpoint}") do
-            bound.handle_accepted(IO::Stream::Buffered.wrap(b), endpoint: endpoint)
-          end
-          engine.handle_connected(IO::Stream::Buffered.wrap(a), endpoint: endpoint)
+          bound.connection_ready(server, endpoint: endpoint)
+          engine.connection_ready(client, endpoint: endpoint)
         end
 
 
@@ -84,7 +86,7 @@ module NNQ
         end
 
 
-        # No accept loop: inproc connects synchronously.
+        # No accept loop: inproc connects are fully synchronous.
         def start_accept_loop(_parent_task, &_on_accepted)
         end
 

data/lib/nnq/transport/ipc.rb

@@ -13,13 +13,16 @@ module NNQ
     # verbatim.
     #
     module IPC
+      Engine.transports["ipc"] = self
+
+
       class << self
         # Binds an IPC server.
         #
         # @param endpoint [String] e.g. "ipc:///tmp/nnq.sock" or "ipc://@abstract"
         # @param engine [Engine]
         # @return [Listener]
-        def bind(endpoint, engine)
+        def bind(endpoint, engine, **)
           path      = parse_path(endpoint)
           sock_path = to_socket_path(path)
 
@@ -35,7 +38,7 @@ module NNQ
         # @param endpoint [String]
         # @param engine [Engine]
         # @return [void]
-        def connect(endpoint, engine)
+        def connect(endpoint, engine, **)
           path      = parse_path(endpoint)
           sock_path = to_socket_path(path)
           sock      = UNIXSocket.new(sock_path)

data/lib/nnq/transport/tcp.rb

@@ -11,13 +11,16 @@ module NNQ
     # accept inside an Async fiber.
     #
     module TCP
+      Engine.transports["tcp"] = self
+
+
       class << self
         # Binds a TCP server to +endpoint+.
         #
         # @param endpoint [String] e.g. "tcp://127.0.0.1:5570" or "tcp://127.0.0.1:0"
         # @param engine [Engine]
         # @return [Listener]
-        def bind(endpoint, engine)
+        def bind(endpoint, engine, **)
           host, port = parse_endpoint(endpoint)
           host       = "0.0.0.0" if host == "*"
           server     = TCPServer.new(host, port)
@@ -34,7 +37,7 @@ module NNQ
         # @param endpoint [String]
         # @param engine [Engine]
         # @return [void]
-        def connect(endpoint, engine)
+        def connect(endpoint, engine, **)
           host, port = parse_endpoint(endpoint)
           sock       = ::Socket.tcp(host, port, connect_timeout: connect_timeout(engine.options))
 

data/lib/nnq/version.rb

@@ -1,5 +1,5 @@
 # frozen_string_literal: true
 
 module NNQ
-  VERSION = "0.6.0"
+  VERSION = "0.7.0"
 end

data/lib/nnq.rb

@@ -1,23 +1,24 @@
 # frozen_string_literal: true
 
 require "protocol/sp"
+require "io/stream"
 
-module NNQ
-  # Freezes module-level state so NNQ sockets can be used inside Ractors.
-  # Call this once before spawning any Ractors that create NNQ sockets.
-  #
-  def self.freeze_for_ractors!
-    Engine::CONNECTION_FAILED.freeze
-    Engine::CONNECTION_LOST.freeze
-    Engine::TRANSPORTS.freeze
-  end
-end
 
+# Core
 require_relative "nnq/version"
-require_relative "nnq/error"
+require_relative "nnq/constants"
+require_relative "nnq/reactor"
 require_relative "nnq/options"
+require_relative "nnq/error"
 require_relative "nnq/connection"
 require_relative "nnq/engine"
+
+# Transport
+require_relative "nnq/transport/inproc"
+require_relative "nnq/transport/tcp"
+require_relative "nnq/transport/ipc"
+
+# Socket types
 require_relative "nnq/socket"
 require_relative "nnq/push_pull"
 require_relative "nnq/pair"

metadata

@@ -1,7 +1,7 @@
 --- !ruby/object:Gem::Specification
 name: nnq
 version: !ruby/object:Gem::Version
-  version: 0.6.0
+  version: 0.7.0
 platform: ruby
 authors:
 - Patrik Wenger
@@ -66,12 +66,12 @@ files:
 - lib/nnq.rb
 - lib/nnq/bus.rb
 - lib/nnq/connection.rb
+- lib/nnq/constants.rb
 - lib/nnq/engine.rb
 - lib/nnq/engine/connection_lifecycle.rb
 - lib/nnq/engine/reconnect.rb
 - lib/nnq/engine/socket_lifecycle.rb
 - lib/nnq/error.rb
-- lib/nnq/monitor_event.rb
 - lib/nnq/options.rb
 - lib/nnq/pair.rb
 - lib/nnq/pub_sub.rb
@@ -97,6 +97,7 @@ files:
 - lib/nnq/socket.rb
 - lib/nnq/surveyor_respondent.rb
 - lib/nnq/transport/inproc.rb
+- lib/nnq/transport/inproc/pipe.rb
 - lib/nnq/transport/ipc.rb
 - lib/nnq/transport/tcp.rb
 - lib/nnq/version.rb

data/lib/nnq/monitor_event.rb

@@ -1,18 +0,0 @@
-# frozen_string_literal: true
-
-module NNQ
-  # Lifecycle event emitted by {Socket#monitor}.
-  #
-  # @!attribute [r] type
-  #   @return [Symbol] event type (:listening, :connected, :disconnected, ...)
-  # @!attribute [r] endpoint
-  #   @return [String, nil] the endpoint involved
-  # @!attribute [r] detail
-  #   @return [Hash, nil] extra context
-  #
-  MonitorEvent = Data.define(:type, :endpoint, :detail) do
-    def initialize(type:, endpoint: nil, detail: nil)
-      super
-    end
-  end
-end