nnq 0.2.0

checksums.yaml

@@ -0,0 +1,7 @@
+---
+SHA256:
+  metadata.gz: 1d4408e132c91a9ebf74a96a8c14f6004c5e05c4b9abdc959a798b98f7adbf32
+  data.tar.gz: ebc0be58942030dcd326664ddde9286d948e074c953ec06f9c238a59b6f3b02a
+SHA512:
+  metadata.gz: dc12b86758f90e1c36aba100962fc89d8cabbc93d72e9e28d4f7973a90cb1b9fab1ab38908cd3988f1b264f3bff2de34818043e5026cdc5ad655312e374fd150
+  data.tar.gz: 7679994454867fa4fcc35d3c16e7c49ffe275ba9cdaf66fbcd428e3fef8b0e89df440918232b0968e38448033198be6f91616aac678ed65bd9c5158a9cda4702

data/CHANGELOG.md

@@ -0,0 +1,27 @@
+# Changelog
+
+## 0.2.0 — 2026-04-09
+
+- `NNQ::PUB` / `NNQ::SUB` with local prefix filtering (pub0/sub0).
+- `NNQ::PAIR` (pair0) — first-pipe-wins exclusive bidirectional channel.
+- `NNQ::REQ` / `NNQ::REP` (req0/rep0) cooked-mode request/reply.
+- `NNQ::Engine::ConnectionLifecycle` — per-connection state machine
+  (`new → handshaking → ready → closed`) consolidating registration,
+  teardown ordering, and idempotent loss handling.
+- `NNQ::Engine::SocketLifecycle` — socket-level state machine
+  (`new → open → closing → closed`) owning the parent task capture and
+  close sequencing.
+- `NNQ::ConnectionRejected` — raised by routing strategies (e.g. PAIR's
+  second peer) to reject a just-handshook connection without exposing
+  it to pumps.
+
+## 0.1.0 — 2026-04-09
+
+Initial Phase 1 slice (push0/pull0 over TCP). Requires Ruby >= 4.0.
+
+- `NNQ::Send::Staging` — opportunistic-batching, HWM-free send path.
+- `NNQ::Connection`, `NNQ::Engine`, `NNQ::Socket`.
+- `NNQ::PUSH` / `NNQ::PULL` with `NNQ::Routing::Push` (round-robin send)
+  and `NNQ::Routing::Pull` (unbounded fair-queue receive).
+- `NNQ::Transport::TCP`.
+- `NNQ::Reactor` — per-process fallback IO thread for non-Async callers.

data/LICENSE

@@ -0,0 +1,13 @@
+Copyright (c) 2026, Patrik Wenger
+
+Permission to use, copy, modify, and/or distribute this software for any
+purpose with or without fee is hereby granted, provided that the above
+copyright notice and this permission notice appear in all copies.
+
+THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES
+WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF
+MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR
+ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
+WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
+ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF
+OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.

data/README.md

@@ -0,0 +1,57 @@
+# NNQ — pure Ruby NNG on Async
+
+[![CI](https://github.com/paddor/nnq/actions/workflows/ci.yml/badge.svg)](https://github.com/paddor/nnq/actions/workflows/ci.yml)
+[![Gem Version](https://img.shields.io/gem/v/nnq?color=e9573f)](https://rubygems.org/gems/nnq)
+[![License: ISC](https://img.shields.io/badge/License-ISC-blue.svg)](LICENSE)
+[![Ruby](https://img.shields.io/badge/Ruby-%3E%3D%204.0-CC342D?logo=ruby&logoColor=white)](https://www.ruby-lang.org)
+
+NNQ is a pure-Ruby implementation of nanomsg's Scalability Protocols
+(SP), wire-compatible with libnng. It is the nng-philosophy sibling of
+[omq](https://github.com/paddor/omq) (pure-Ruby ZeroMQ).
+
+Status: pre-alpha. v0.0.1 implements push0/pull0 over TCP only. See
+[PLAN.md](PLAN.md) for the design and roadmap.
+
+## Why a pure-Ruby NNG?
+
+- **No native deps.** Same stack as omq: `async`, `io-stream`,
+  `protocol-sp`. No C extension, no FFI.
+- **Faster than libnng** for the multi-fiber case (target: 10–25×).
+  libnng's per-message aio model leaves all the throughput of write
+  coalescing on the table.
+- **Async-native.** Wrap in `Async{}`, no background thread for users
+  who already run a reactor.
+
+## Quickstart
+
+```ruby
+require "nnq"
+require "async"
+
+Async do
+  pull = NNQ::PULL.bind("tcp://127.0.0.1:5570")
+  push = NNQ::PUSH.connect("tcp://127.0.0.1:5570")
+
+  push.send("hello")
+  puts pull.receive  # => "hello"
+
+  push.close
+  pull.close
+end
+```
+
+## The "no HWM" philosophy
+
+NNQ has no high-water mark. Backpressure comes only from the kernel
+socket buffer. Concurrent senders are coalesced into one writev syscall
+by `NNQ::Send::Staging`, which keeps a tiny in-memory aggregation point
+that exists only during the moments when the drainer fiber is parked
+on `wait_writable`. See `lib/nnq/send/staging.rb` for the full
+discussion.
+
+## Cancellation
+
+`Async::Task` is the aio. To cancel an in-flight send, stop the task
+that called `send`. The semantics match libnng's
+`nng_aio_cancel` exactly: best-effort, may lose the race if the message
+is already on the wire.

data/lib/nnq/connection.rb

@@ -0,0 +1,90 @@
+# frozen_string_literal: true
+
+require "protocol/sp"
+
+module NNQ
+  # Per-pipe state: thin wrapper around Protocol::SP::Connection.
+  #
+  # Owns no fibers itself — recv loop and send pump are spawned by
+  # the Engine and routing strategy respectively.
+  #
+  class Connection
+    # @return [Protocol::SP::Connection]
+    attr_reader :sp
+
+    # @return [String, nil] endpoint URI we connected to / accepted from
+    attr_reader :endpoint
+
+    # @param sp [Protocol::SP::Connection] handshake-completed SP connection
+    # @param endpoint [String, nil]
+    def initialize(sp, endpoint: nil)
+      @sp       = sp
+      @endpoint = endpoint
+      @closed   = false
+    end
+
+
+    # @return [Integer] peer protocol id (e.g. Protocols::PULL_V0)
+    def peer_protocol = @sp.peer_protocol
+
+
+    # Writes one message into the SP connection's send buffer (no flush).
+    #
+    # @param body [String]
+    # @return [void]
+    def write_message(body)
+      raise ClosedError, "connection closed" if @closed
+      @sp.write_message(body)
+    end
+
+
+    # Writes a batch of bodies under a single SP mutex acquisition.
+    # Used by the work-stealing send pump hot path.
+    #
+    # @param bodies [Array<String>]
+    # @return [void]
+    def write_messages(bodies)
+      raise ClosedError, "connection closed" if @closed
+      @sp.write_messages(bodies)
+    end
+
+
+    # Writes one message AND flushes immediately. Used by REQ/REP where
+    # each call is request-paced and there's nothing to batch.
+    #
+    # @param body [String]
+    # @return [void]
+    def send_message(body)
+      raise ClosedError, "connection closed" if @closed
+      @sp.send_message(body)
+    end
+
+
+    # Flushes the SP connection's send buffer to the socket.
+    #
+    # @return [void]
+    def flush
+      @sp.flush
+    end
+
+
+    # Reads one message body off the wire. Blocks the calling fiber.
+    #
+    # @return [String]
+    def receive_message
+      @sp.receive_message
+    end
+
+
+    # @return [Boolean]
+    def closed? = @closed
+
+
+    # Closes the underlying SP connection. Safe to call twice.
+    def close
+      return if @closed
+      @closed = true
+      @sp.close
+    end
+  end
+end

data/lib/nnq/engine/connection_lifecycle.rb

@@ -0,0 +1,132 @@
+# frozen_string_literal: true
+
+require "protocol/sp"
+require_relative "../connection"
+
+module NNQ
+  class Engine
+    # Owns the full arc of one connection: handshake → ready → closed.
+    #
+    # Centralizes the ordering of side effects (routing registration,
+    # teardown) so the sequence lives in one place instead of being
+    # scattered across Engine, the accept/connect paths, and the
+    # recv/send pumps.
+    #
+    # State machine:
+    #
+    #     new → handshaking → ready → closed
+    #
+    # #lost! and #close! are idempotent — the state guard ensures side
+    # effects run exactly once even if multiple pumps race to report a
+    # lost connection.
+    #
+    class ConnectionLifecycle
+      class InvalidTransition < RuntimeError; end
+
+      STATES = %i[new handshaking ready closed].freeze
+
+      TRANSITIONS = {
+        new:         %i[handshaking ready closed].freeze,
+        handshaking: %i[ready closed].freeze,
+        ready:       %i[closed].freeze,
+        closed:      [].freeze,
+      }.freeze
+
+
+      # @return [NNQ::Connection, nil]
+      attr_reader :conn
+
+      # @return [String, nil]
+      attr_reader :endpoint
+
+      # @return [Symbol]
+      attr_reader :state
+
+
+      # @param engine [Engine]
+      # @param endpoint [String, nil]
+      # @param framing [Symbol] :tcp or :ipc
+      def initialize(engine, endpoint:, framing:)
+        @engine   = engine
+        @endpoint = endpoint
+        @framing  = framing
+        @state    = :new
+        @conn     = nil
+      end
+
+
+      # Performs the SP handshake, wraps the result in NNQ::Connection,
+      # registers with the engine and routing, and transitions to :ready.
+      #
+      # @param io [#read, #write, #close]
+      # @return [NNQ::Connection]
+      def handshake!(io)
+        transition!(:handshaking)
+        sp = Protocol::SP::Connection.new(
+          io,
+          protocol:         @engine.protocol,
+          max_message_size: @engine.options.max_message_size,
+          framing:          @framing,
+        )
+        sp.handshake!
+        ready!(NNQ::Connection.new(sp, endpoint: @endpoint))
+        @conn
+      rescue
+        io.close rescue nil
+        transition!(:closed) unless @state == :closed
+        raise
+      end
+
+
+      # Transitions to :closed, removing the connection from the engine
+      # and notifying the routing strategy. Idempotent.
+      def lost!
+        tear_down!
+      end
+
+
+      # Alias for lost!. Kept as a separate method for parity with OMQ,
+      # where the distinction drives reconnect scheduling. nnq has no
+      # reconnect yet, so the two behave identically.
+      def close!
+        tear_down!
+      end
+
+
+      private
+
+      def ready!(conn)
+        @conn                    = conn
+        @engine.connections[conn] = self
+        transition!(:ready)
+        begin
+          @engine.routing.connection_added(conn) if @engine.routing.respond_to?(:connection_added)
+        rescue ConnectionRejected
+          tear_down!
+          raise
+        end
+        @engine.new_pipe.signal
+      end
+
+
+      def tear_down!
+        return if @state == :closed
+        transition!(:closed)
+        if @conn
+          @engine.connections.delete(@conn)
+          @engine.routing.connection_removed(@conn) if @engine.routing.respond_to?(:connection_removed)
+          @conn.close rescue nil
+        end
+      end
+
+
+      def transition!(new_state)
+        allowed = TRANSITIONS[@state]
+        unless allowed&.include?(new_state)
+          raise InvalidTransition, "#{@state} → #{new_state}"
+        end
+        @state = new_state
+      end
+    end
+  end
+end

data/lib/nnq/engine/socket_lifecycle.rb

@@ -0,0 +1,88 @@
+# frozen_string_literal: true
+
+module NNQ
+  class Engine
+    # Owns the socket-level state: `:new → :open → :closing → :closed`
+    # and the captured parent task for the socket's task tree.
+    #
+    # Engine delegates state queries here and uses it to coordinate the
+    # ordering of close-time side effects. Mirrors OMQ's SocketLifecycle
+    # without the heartbeat/mechanism/monitor machinery nnq doesn't need.
+    #
+    class SocketLifecycle
+      class InvalidTransition < RuntimeError; end
+
+      STATES = %i[new open closing closed].freeze
+
+      TRANSITIONS = {
+        new:     %i[open closed].freeze,
+        open:    %i[closing closed].freeze,
+        closing: %i[closed].freeze,
+        closed:  [].freeze,
+      }.freeze
+
+
+      # @return [Symbol]
+      attr_reader :state
+
+      # @return [Async::Task, nil] root of the socket's task tree
+      attr_reader :parent_task
+
+      # @return [Boolean] true if parent_task is the shared Reactor thread
+      attr_reader :on_io_thread
+
+
+      def initialize
+        @state        = :new
+        @parent_task  = nil
+        @on_io_thread = false
+      end
+
+
+      def open?    = @state == :open
+      def closing? = @state == :closing
+      def closed?  = @state == :closed
+      def alive?   = @state == :new || @state == :open
+
+
+      # Captures +task+ as this socket's task tree root. Transitions
+      # `:new → :open`. Idempotent: second call is a no-op.
+      #
+      # @param task [Async::Task]
+      # @param on_io_thread [Boolean] true when +task+ is the shared
+      #   NNQ::Reactor root task (vs. the caller's own Async task)
+      # @return [Boolean] true on first-time capture, false if already captured
+      def capture_parent_task(task, on_io_thread:)
+        return false if @parent_task
+        @parent_task  = task
+        @on_io_thread = on_io_thread
+        transition!(:open)
+        true
+      end
+
+
+      # Transitions `:open → :closing`.
+      def start_closing!
+        transition!(:closing)
+      end
+
+
+      # Transitions `:closing → :closed` (or `:new → :closed` for
+      # never-opened sockets).
+      def finish_closing!
+        transition!(:closed)
+      end
+
+
+      private
+
+      def transition!(new_state)
+        allowed = TRANSITIONS[@state]
+        unless allowed&.include?(new_state)
+          raise InvalidTransition, "#{@state} → #{new_state}"
+        end
+        @state = new_state
+      end
+    end
+  end
+end

data/lib/nnq/engine.rb

@@ -0,0 +1,198 @@
+# frozen_string_literal: true
+
+require "async"
+require "async/clock"
+require "protocol/sp"
+require_relative "error"
+require_relative "connection"
+require_relative "reactor"
+require_relative "engine/socket_lifecycle"
+require_relative "engine/connection_lifecycle"
+require_relative "transport/tcp"
+require_relative "transport/ipc"
+require_relative "transport/inproc"
+
+module NNQ
+  # Per-socket orchestrator. Owns the listener set, the connection map
+  # (keyed on NNQ::Connection, with per-connection ConnectionLifecycle
+  # values), the transport registry, and the socket-level state machine
+  # via {SocketLifecycle}.
+  #
+  # Mirrors OMQ's Engine in shape but is much smaller because there's
+  # no HWM bookkeeping, no mechanisms, no heartbeat, no monitor queue.
+  #
+  class Engine
+    TRANSPORTS = {
+      "tcp"    => Transport::TCP,
+      "ipc"    => Transport::IPC,
+      "inproc" => Transport::Inproc,
+    }
+
+
+    # @return [Integer] our SP protocol id (e.g. Protocols::PUSH_V0)
+    attr_reader :protocol
+
+    # @return [Options]
+    attr_reader :options
+
+    # @return [Hash{NNQ::Connection => ConnectionLifecycle}]
+    attr_reader :connections
+
+    # @return [SocketLifecycle]
+    attr_reader :lifecycle
+
+    # @return [String, nil]
+    attr_reader :last_endpoint
+
+    # @return [Async::Condition] signaled when a new pipe is registered
+    attr_reader :new_pipe
+
+
+    # @param protocol [Integer] our SP protocol id (e.g. Protocols::PUSH_V0)
+    # @param options [Options]
+    # @yieldparam engine [Engine] used by the caller to build a routing
+    #   strategy with access to the engine's connection map
+    def initialize(protocol:, options:)
+      @protocol      = protocol
+      @options       = options
+      @connections   = {}
+      @listeners     = []
+      @lifecycle     = SocketLifecycle.new
+      @last_endpoint = nil
+      @new_pipe      = Async::Condition.new
+      @routing       = yield(self)
+    end
+
+
+    # @return [Routing strategy]
+    attr_reader :routing
+
+
+    # @return [Async::Task, nil]
+    def parent_task = @lifecycle.parent_task
+
+
+    def closed? = @lifecycle.closed?
+
+
+    # Stores the parent Async task that long-lived NNQ fibers will
+    # attach to. The caller (Socket) is responsible for picking the
+    # right one (the user's current task, or Reactor.root_task).
+    def capture_parent_task(task)
+      on_io_thread = task.equal?(Reactor.root_task)
+      @lifecycle.capture_parent_task(task, on_io_thread: on_io_thread)
+    end
+
+
+    # Binds to +endpoint+. Synchronous: errors propagate.
+    def bind(endpoint)
+      transport = transport_for(endpoint)
+      listener  = transport.bind(endpoint, self)
+      listener.start_accept_loop(@lifecycle.parent_task) do |io, framing = :tcp|
+        handle_accepted(io, endpoint: endpoint, framing: framing)
+      end
+      @listeners << listener
+      @last_endpoint = listener.endpoint
+    end
+
+
+    # Connects to +endpoint+. Synchronous on first attempt; reconnect
+    # is wired in Phase 1.1.
+    def connect(endpoint)
+      transport = transport_for(endpoint)
+      transport.connect(endpoint, self)
+      @last_endpoint = endpoint
+    rescue Errno::ECONNREFUSED, Errno::EHOSTUNREACH => e
+      raise Error, "could not connect to #{endpoint}: #{e.class}: #{e.message}"
+    end
+
+
+    # Called by transports for each accepted client connection.
+    def handle_accepted(io, endpoint:, framing: :tcp)
+      lifecycle = ConnectionLifecycle.new(self, endpoint: endpoint, framing: framing)
+      lifecycle.handshake!(io)
+      spawn_recv_loop(lifecycle.conn) if @routing.respond_to?(:enqueue) && @connections.key?(lifecycle.conn)
+    rescue ConnectionRejected
+      # routing rejected this peer (e.g. PAIR already bonded) — lifecycle cleaned up
+    rescue => e
+      warn("nnq: handshake failed for #{endpoint}: #{e.class}: #{e.message}") if $DEBUG
+    end
+
+
+    # Called by transports for each dialed connection.
+    def handle_connected(io, endpoint:, framing: :tcp)
+      lifecycle = ConnectionLifecycle.new(self, endpoint: endpoint, framing: framing)
+      lifecycle.handshake!(io)
+      spawn_recv_loop(lifecycle.conn) if @routing.respond_to?(:enqueue) && @connections.key?(lifecycle.conn)
+    rescue ConnectionRejected
+      # unusual on connect side, but handled identically
+    end
+
+
+    # Spawns a task under the socket's parent task. Used by routing
+    # strategies (e.g. PUSH send pump) to attach long-lived fibers to
+    # the engine's lifecycle without going through transient: true.
+    def spawn_task(annotation:, &block)
+      @lifecycle.parent_task.async(annotation: annotation, &block)
+    end
+
+
+    # Closes the engine: stops listeners, drains the send queue subject
+    # to linger, stops routing pumps (which by now are parked on the
+    # empty queue), then tears down every connection's lifecycle. Order
+    # matters — closing connections first would force mid-flush pumps
+    # to abort with IOError.
+    def close
+      return unless @lifecycle.alive?
+      @lifecycle.start_closing!
+      @listeners.each(&:stop)
+      drain_send_queue(@options.linger)
+      @routing.close if @routing.respond_to?(:close)
+      # Tear down each remaining connection via its lifecycle. The
+      # collection mutates during iteration, so snapshot the values.
+      @connections.values.each(&:close!)
+      @lifecycle.finish_closing!
+      @new_pipe.signal
+    end
+
+
+    # Called by routing pumps (or the recv loop) when their connection
+    # has died. Idempotent via the lifecycle state guard.
+    def handle_connection_lost(conn)
+      @connections[conn]&.lost!
+    end
+
+
+    private
+
+    def drain_send_queue(timeout)
+      return unless @routing.respond_to?(:send_queue_drained?)
+      return if @connections.empty?
+      deadline = timeout ? Async::Clock.now + timeout : nil
+      until @routing.send_queue_drained?
+        break if deadline && (deadline - Async::Clock.now) <= 0
+        sleep 0.001
+      end
+    end
+
+
+    def spawn_recv_loop(conn)
+      @lifecycle.parent_task.async(annotation: "nnq recv #{conn.endpoint}") do
+        loop do
+          body = conn.receive_message
+          @routing.enqueue(body, conn)
+        rescue EOFError, IOError, Errno::ECONNRESET, Async::Stop
+          break
+        end
+      ensure
+        handle_connection_lost(conn)
+      end
+    end
+
+
+    def transport_for(endpoint)
+      scheme = endpoint[/\A([a-z+]+):\/\//i, 1] or raise Error, "no scheme: #{endpoint}"
+      TRANSPORTS[scheme] or raise Error, "unsupported transport: #{scheme}"
+    end
+  end
+end

data/lib/nnq/error.rb

@@ -0,0 +1,10 @@
+# frozen_string_literal: true
+
+module NNQ
+  class Error           < RuntimeError; end
+  class ClosedError     < Error; end
+  class ProtocolError   < Error; end
+  class TimeoutError    < Error; end
+  class RequestCancelled < Error; end
+  class ConnectionRejected < Error; end
+end

data/lib/nnq/options.rb

@@ -0,0 +1,31 @@
+# frozen_string_literal: true
+
+module NNQ
+  # Per-socket configuration. Deliberately tiny — no conflate, no heartbeat
+  # command (nng uses TCP keepalive instead). New options should match nng
+  # socket option names where they exist (`recv_max_size`, `reconnect_time`,
+  # `send_buf`, etc.).
+  #
+  # `send_hwm` bounds *one shared queue per socket*, not per peer. See
+  # DESIGN.md "Per-socket HWM".
+  #
+  class Options
+    DEFAULT_HWM = 1000
+
+    attr_accessor :linger
+    attr_accessor :read_timeout
+    attr_accessor :write_timeout
+    attr_accessor :reconnect_interval
+    attr_accessor :max_message_size
+    attr_accessor :send_hwm
+
+    def initialize(linger: nil, send_hwm: DEFAULT_HWM)
+      @linger             = linger
+      @read_timeout       = nil
+      @write_timeout      = nil
+      @reconnect_interval = 0.1
+      @max_message_size   = nil
+      @send_hwm           = send_hwm
+    end
+  end
+end

data/lib/nnq/pair.rb

@@ -0,0 +1,33 @@
+# frozen_string_literal: true
+
+require_relative "socket"
+require_relative "routing/pair"
+
+module NNQ
+  # PAIR (nng pair0): exclusive bidirectional channel with a single
+  # peer. First peer to connect wins; subsequent peers are dropped
+  # until the current one disconnects. No SP header on the wire.
+  #
+  class PAIR < Socket
+    def send(body)
+      Reactor.run { @engine.routing.send(body) }
+    end
+
+
+    def receive
+      Reactor.run { @engine.routing.receive }
+    end
+
+
+    private
+
+    def protocol
+      Protocol::SP::Protocols::PAIR_V0
+    end
+
+
+    def build_routing(engine)
+      Routing::Pair.new(engine)
+    end
+  end
+end