nnq-zstd 0.1.0

checksums.yaml

@@ -0,0 +1,7 @@
+---
+SHA256:
+  metadata.gz: c2f3392c672cfc2022eee048702a0211999e58fd0e723bc50f0ae1c789a114d9
+  data.tar.gz: de1740ec0a048c8bbb099ccfdefba9cae7103c022ea06c9d335121aa25c63ece
+SHA512:
+  metadata.gz: bc58535bbe51850ca68b3f7fc24f7bcaf7a154ebb0b90730ad4968e8ad845123c84b6b62c6776e6e286d4fe866c55db45c969b1d33c747af5ff0daa5ab76fb8b
+  data.tar.gz: 1f380726e3ed288d9b6502b748259cd4f8d1afcbc31910931410eafbb5e778e74c614caea631e04f1a2ea34ebdb8b34bbdeecf45849a221989cd0e50bc573f7b

data/CHANGELOG.md

@@ -0,0 +1,20 @@
+# Changelog
+
+## 0.1.0 — unreleased
+
+Initial release.
+
+- `NNQ::Zstd.wrap(socket, level: -3, dict: nil)` — transparent Zstd
+  compression decorator around an `NNQ::Socket`.
+- Sender-side dictionary training: first ~1000 messages < 1 KiB each,
+  or up to 100 KiB cumulative sample bytes. Training failure disables
+  training for the session.
+- In-band dict shipping via a 4-byte preamble discriminator
+  (`00 00 00 00` = plaintext, Zstd frame magic = compressed, Zstd
+  dict magic = dictionary).
+- Bounded decompression: `Frame_Content_Size` required; effective
+  cap is `min(16 MiB, socket.recv_maxsz)`.
+- Per-wrapper caps: 32 dicts, 128 KiB cumulative. Violations raise
+  `NNQ::Zstd::ProtocolError`.
+- Auto-generated dict IDs restricted to the user range
+  `32_768..(2**31 - 1)`.

data/LICENSE

@@ -0,0 +1,13 @@
+Copyright (c) 2026, Patrik Wenger
+
+Permission to use, copy, modify, and/or distribute this software for any
+purpose with or without fee is hereby granted, provided that the above
+copyright notice and this permission notice appear in all copies.
+
+THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES
+WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF
+MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR
+ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
+WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
+ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF
+OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.

data/README.md

@@ -0,0 +1,63 @@
+# NNQ::Zstd — transparent Zstd compression for NNQ sockets
+
+[![CI](https://github.com/paddor/nnq-zstd/actions/workflows/ci.yml/badge.svg)](https://github.com/paddor/nnq-zstd/actions/workflows/ci.yml)
+[![Gem Version](https://img.shields.io/gem/v/nnq-zstd?color=e9573f)](https://rubygems.org/gems/nnq-zstd)
+[![License: ISC](https://img.shields.io/badge/License-ISC-blue.svg)](LICENSE)
+[![Ruby](https://img.shields.io/badge/Ruby-%3E%3D%204.0-CC342D?logo=ruby&logoColor=white)](https://www.ruby-lang.org)
+
+Wraps any `NNQ::Socket` with transparent per-message Zstd compression,
+sender-side dictionary training, and in-band dictionary shipping. No
+handshake, no negotiation — both peers must wrap for it to work.
+
+See [RFC.md](RFC.md) for the normative wire protocol.
+
+## Quick start
+
+```ruby
+require "nnq"
+require "nnq/zstd"
+
+push = NNQ::PUSH0.new
+push.connect("tcp://127.0.0.1:5555")
+push = NNQ::Zstd.wrap(push, level: -3)   # fast (-3) or balanced (3)
+
+push.send("payload")                      # compressed on the wire
+```
+
+```ruby
+pull = NNQ::PULL0.new
+pull.bind("tcp://*:5555")
+pull = NNQ::Zstd.wrap(pull)               # receiver-only: no dict config
+
+pull.receive                              # => "payload"
+```
+
+## How it works
+
+- Every message carries a 4-byte preamble:
+  - `00 00 00 00` — uncompressed plaintext (preamble stripped on recv).
+  - Zstd frame magic — a full Zstd frame (compressed).
+  - Zstd dict magic — a Zstd dictionary to install (not surfaced to the app).
+- The sender trains a dict from its first ~1000 small messages (or 100 KiB
+  cumulative sample bytes), then compresses subsequent small messages
+  with it. The dict is shipped in-band before the next real payload.
+- User-supplied dictionaries may be passed via `dict:`:
+  ```ruby
+  NNQ::Zstd.wrap(socket, level: -3, dict: [dict_bytes_1, dict_bytes_2])
+  ```
+  All supplied dicts are shipped to peers on the wire. Training is
+  skipped when `dict:` is given.
+- Decompression is bounded by `min(16 MiB, recv_maxsz)`. Frames whose
+  header omits `Frame_Content_Size` are rejected.
+- Up to **32 dicts** and **128 KiB cumulative** per wrapper. Any
+  protocol violation raises `NNQ::Zstd::ProtocolError`.
+
+## Out of scope
+
+- Negotiation / auto-detection. Both peers must wrap.
+- Dict persistence across process restarts. Training is per-session.
+- Receiver-side training — receivers only install shipped dicts.
+
+## License
+
+ISC. See [LICENSE](LICENSE).

data/RFC.md

@@ -0,0 +1,283 @@
+# NNQ-Zstd wire protocol (RFC-style)
+
+**Status:** experimental. Version: 0.1.0.
+
+## 1. Scope and non-goals
+
+This document specifies the wire format and behavior of a transparent
+Zstandard compression layer for messages carried over NNG/SP sockets
+(`inproc`, `ipc`, `tcp`). It is not a replacement for SP and does not
+define a connection-level handshake.
+
+Non-goals:
+
+- No capability negotiation. Both peers MUST wrap their sockets; an
+  unwrapped peer will see garbled bytes and SHOULD disconnect.
+- No persistence of dictionaries across process restarts.
+- No receiver-side training. Receivers only install dictionaries
+  shipped by senders.
+
+The key words **MUST**, **MUST NOT**, **SHOULD**, **MAY**, and
+**SHALL** are to be interpreted as described in RFC 2119.
+
+## 2. Terminology
+
+- **Sender** — a wrapper instance that sends messages.
+- **Receiver** — a wrapper instance that receives messages.
+- **Wrapper** — an `NNQ::Zstd::Wrapper` (or equivalent in another
+  language) decorating an `NNQ::Socket`.
+- **Codec** — the pure state machine inside the wrapper that
+  encodes/decodes individual messages.
+- **Dictionary** (**dict**) — a Zstandard dictionary, as produced by
+  `ZDICT_trainFromBuffer` or handed in by the user.
+- **Dict ID** — the 32-bit `Dictionary_ID` field carried in the
+  dict's header and in each frame header that references it.
+- **Preamble** — the first four bytes of every wire message; see §3.
+- **Wire message** — one SP message as handed to/from the underlying
+  socket.
+
+## 3. Wire format
+
+Every wire message begins with a 4-byte preamble. The first four
+bytes discriminate three mutually exclusive cases:
+
+| First 4 bytes (hex, wire order) | LE u32      | Meaning       |
+|---|---|---|
+| `00 00 00 00`                   | `0x00000000` | uncompressed  |
+| `28 B5 2F FD`                   | `0xFD2FB528` | Zstd frame    |
+| `37 A4 30 EC`                   | `0xEC30A437` | Zstd dict     |
+
+The Zstandard frame magic and Zstandard dictionary magic are fixed
+by the Zstandard format specification. The NUL preamble is specific
+to this protocol.
+
+### 3.1 Uncompressed
+
+```
++----+----+----+----+=========================+
+| 00 | 00 | 00 | 00 |   plaintext body        |
++----+----+----+----+=========================+
+```
+
+The receiver MUST strip the 4-byte NUL preamble and deliver the
+remainder to the application.
+
+### 3.2 Zstd frame
+
+```
++----+----+----+----+=========================+
+| 28 | B5 | 2F | FD |   rest of Zstd frame    |
++----+----+----+----+=========================+
+```
+
+The entire wire message is a valid Zstandard frame; the preamble
+is the frame's magic number. The receiver MUST NOT strip the
+preamble before decompression.
+
+### 3.3 Zstd dictionary
+
+```
++----+----+----+----+=========================+
+| 37 | A4 | 30 | EC |   rest of Zstd dict     |
++----+----+----+----+=========================+
+```
+
+The entire wire message is a valid Zstandard dictionary. The
+receiver MUST install the dictionary in its local store (§5.3)
+and MUST NOT deliver this message to the application.
+
+## 4. Sender behavior
+
+### 4.1 Levels
+
+The sender's compression level is set at wrapper construction. This
+specification RECOMMENDS two levels:
+
+- `-3` — fast strategy, low CPU, moderate ratio.
+- `3` — balanced Zstd default.
+
+Implementations MAY support arbitrary Zstd levels.
+
+### 4.2 Training
+
+Unless the user supplied dictionaries at construction, the sender
+SHALL collect training samples from outbound messages until either
+condition is met:
+
+- **1000** samples collected, OR
+- **100 KiB** cumulative sample bytes collected.
+
+Only messages with body size **< 1024 bytes** are eligible as
+samples. The sender SHALL train via ZDICT with a dictionary
+capacity of **8 KiB** (the training cap; the resulting dict may be
+smaller).
+
+On training failure (insufficient or pathological samples, ZDICT
+internal error), the sender SHALL permanently disable training for
+the session and continue in no-dict mode.
+
+### 4.3 Auto-generated dictionary IDs
+
+Auto-trained dictionary IDs MUST fall in the user range:
+
+```
+USER_DICT_ID_RANGE = 32_768 .. (2**31 - 1)
+```
+
+Values `0..32767` are reserved for a future registrar; values
+`>= 2**31` are reserved by the Zstandard format. Senders MUST NOT
+assign auto-trained dicts to either reserved range. This is
+enforced by post-patching bytes `[4..7]` of the trained dict buffer
+with a randomly-chosen u32 LE value in the user range.
+
+User-supplied dictionaries passed via `dict:` are honored as-is.
+The user is responsible for choosing a non-reserved ID, or for
+accepting the collision risk of using a reserved one.
+
+### 4.4 Compression thresholds
+
+- **No dict loaded**: the sender MUST emit the message uncompressed
+  (§3.1) if `body.bytesize < 512`.
+- **With a loaded dict**: the sender MUST emit the message
+  uncompressed if `body.bytesize < 64`.
+
+In addition, the sender MUST emit the message uncompressed if the
+compressed result would not save at least four bytes (i.e.
+`compressed.bytesize >= body.bytesize - 4`); this avoids paying a
+preamble's worth of overhead for negative wins.
+
+### 4.5 Dictionary shipping
+
+**Every dictionary the sender knows** (user-supplied or
+auto-trained) MUST be delivered to the peer in-band, as a dict
+frame (§3.3), before any payload that requires it.
+
+Shipping policy: a sender SHALL ship all known dicts eagerly. In
+practice this means:
+
+- On the first call to `encode` after construction (or after
+  training completes), return a dict frame for each dict not yet
+  shipped, followed by the real payload.
+- On each newly-connected peer (observed via the underlying
+  socket's monitor stream), re-ship the full known-dict set.
+- **Ordering**: any dict needed to decompress a given payload MUST
+  appear strictly before that payload on the wire.
+
+### 4.6 Per-wrapper caps (sender)
+
+- At most **32 distinct dictionaries**.
+- At most **128 KiB** cumulative dictionary bytes (sum of
+  `dict.bytesize` across the store).
+- There is no per-dictionary size cap beyond the total-bytes
+  budget; a single 128 KiB dict is valid.
+
+If training or user input would exceed either cap, the sender MUST
+refuse to install the offending dict.
+
+### 4.7 Receive-only wrappers
+
+A wrapper that is never asked to `send` naturally skips training
+and dict shipping, since both are driven by outbound traffic. No
+special mode is required: to obtain a decode-only decorator,
+simply `wrap` the socket and only call `receive`.
+
+## 5. Receiver behavior
+
+### 5.1 Preamble dispatch
+
+For each wire message received:
+
+- If it begins with `00 00 00 00`, strip the preamble and deliver
+  the remainder to the application.
+- If it begins with the Zstd frame magic, decompress it per §5.2
+  and deliver the plaintext.
+- If it begins with the Zstd dict magic, install it per §5.3 and
+  do not deliver anything to the application; continue with the
+  next wire message.
+- Otherwise, the preamble is unrecognized. The receiver MUST raise
+  a protocol error and SHOULD treat the wrapped socket as failed.
+
+### 5.2 Bounded decompression
+
+For each Zstd frame:
+
+1. Inspect the frame header for `Frame_Content_Size`. If absent,
+   the receiver MUST raise a protocol error. This is the
+   anti-zip-bomb guarantee.
+2. Read the frame header's `Dictionary_ID` field. If non-zero, the
+   receiver MUST look up a dictionary with that ID in its local
+   store; if absent, raise a protocol error. If zero, decompress
+   without a dictionary.
+3. Call Zstandard decompression with a `max_output_size` equal to
+   `min(16 MiB, recv_maxsz)` where `recv_maxsz` is the wrapped
+   socket's own configured maximum inbound message size (an
+   implementation-defined default applies if the socket has no
+   such cap). Exceeding this size is a protocol error.
+4. Deliver the decompressed plaintext to the application.
+
+### 5.3 Dictionary installation
+
+For each dict frame:
+
+1. Parse the dictionary (the Zstd dict header carries the dict_id
+   at bytes `[4..7]`).
+2. Check the per-wrapper caps (§5.4). A violation is a protocol
+   error.
+3. Insert `id => dictionary` into the local store. If an entry
+   with the same id already exists, overwrite it (idempotent);
+   adjust the cumulative-bytes accounting accordingly.
+4. Do not deliver the dict frame to the application.
+
+Note: dict IDs in the reserved ranges are accepted if a peer
+chooses to ship them (the reserved-range rule applies only to
+auto-generated IDs at the sender; a receiver does not second-guess
+the peer's choices).
+
+### 5.4 Per-wrapper caps (receiver)
+
+- At most **32 distinct dictionaries**.
+- At most **128 KiB** cumulative dictionary bytes.
+
+These match the sender caps and protect against a malicious or
+buggy peer shipping unbounded dictionary state.
+
+### 5.5 `#receive` contract
+
+The wrapper's `#receive` operation MUST NEVER return a dict frame
+to the application. It MUST loop internally over the underlying
+socket's `receive`, silently installing any dict frames it
+encounters, until it produces a real payload (plaintext or
+successfully-decompressed frame) or the underlying socket closes.
+
+## 6. Interoperability
+
+Both peers of a connection MUST wrap their sockets with a
+compatible implementation of this protocol. An unwrapped peer will
+see byte sequences it cannot parse (an SP message starting with a
+NUL preamble, or a valid Zstd frame) and is expected to close the
+connection.
+
+## 7. Security considerations
+
+- **Zip bombs**. §5.2's mandatory `Frame_Content_Size` check plus
+  bounded `max_output_size` prevent an attacker from forcing
+  unbounded memory allocation during decompression.
+- **Dictionary DoS**. §5.4's count and cumulative-bytes caps
+  prevent a malicious peer from exhausting memory by shipping
+  unbounded dictionaries.
+- **Reserved-range squatting**. Auto-trained dicts MUST avoid
+  reserved ID ranges (§4.3) so that future registry-allocated
+  dicts can coexist with in-the-wild private dicts without
+  collision.
+- **No confidentiality or integrity**. This protocol provides
+  neither. Wrap the underlying transport with TLS or a similar
+  mechanism for either property.
+
+## 8. Appendix: test vectors
+
+*(Placeholder — to be filled with concrete hex dumps once the
+reference implementation produces them.)*
+
+1. **NUL-preamble `"hi"`** — wire bytes: `00 00 00 00 68 69`.
+2. **Empty compressed frame** — TBD.
+3. **Minimal trained dictionary** — TBD.

data/lib/nnq/zstd/codec.rb

@@ -0,0 +1,315 @@
+# frozen_string_literal: true
+
+module NNQ
+  module Zstd
+    # Pure state machine: no socket references. `encode(body)` returns
+    # `[wire, dict_frames]` where `dict_frames` are any dict frames
+    # that MUST precede the wire payload on the wire. `decode(wire)`
+    # returns a plaintext String, or `nil` if the wire message was a
+    # dict frame that has been silently installed into the receive-
+    # side dictionary store.
+    class Codec
+      MAX_DECOMPRESSED_SIZE  = 16 * 1024 * 1024
+      MAX_DICTS              = 32
+      MAX_DICTS_TOTAL_BYTES  = 128 * 1024
+      DICT_CAPACITY          = 8 * 1024
+      TRAIN_MAX_SAMPLES      = 1000
+      TRAIN_MAX_BYTES        = 100 * 1024
+      TRAIN_MAX_SAMPLE_LEN   = 1024
+      MIN_COMPRESS_NO_DICT   = 512
+      MIN_COMPRESS_WITH_DICT = 64
+
+      NUL_PREAMBLE = ("\x00" * 4).b.freeze
+      ZSTD_MAGIC   = "\x28\xB5\x2F\xFD".b.freeze
+      ZDICT_MAGIC  = "\x37\xA4\x30\xEC".b.freeze
+
+
+      def initialize(level:, dicts: [], recv_max_size: nil)
+        @level           = level
+        @recv_max_size   = [recv_max_size || MAX_DECOMPRESSED_SIZE, MAX_DECOMPRESSED_SIZE].min
+
+        @send_dicts      = {}
+        @send_dict_bytes = {}
+        @send_dict_order = []
+        @pending_ship    = []
+        @shipped_peers   = Set.new
+        @active_send_id  = nil
+
+        @recv_dicts      = {}
+        @recv_total_bytes = 0
+
+        @training        = dicts.empty?
+        @train_samples   = []
+        @train_bytes     = 0
+
+        dicts.each { |db| install_send_dict(db.b) }
+      end
+
+
+      # @return [Integer, nil] id of the dict currently used for compression
+      def active_send_dict_id
+        @active_send_id
+      end
+
+
+      # @return [Array<Integer>] ids of dicts in the send-side store
+      def send_dict_ids
+        @send_dict_order.dup
+      end
+
+
+      # @return [Array<Integer>] ids of dicts in the recv-side store
+      def recv_dict_ids
+        @recv_dicts.keys
+      end
+
+
+      # Resets the shipped-tracker so the next encode calls will re-emit
+      # every known dict before the next real payload. Called by the
+      # wrapper when a new peer connects.
+      def requeue_all_dicts_for_shipping!
+        @pending_ship = @send_dict_order.dup
+      end
+
+
+      # Encodes `body` into a wire message. Returns `[wire, dict_frames]`
+      # where `dict_frames` is an array of wire messages that MUST be
+      # sent strictly before `wire`.
+      def encode(body)
+        body = body.b
+        maybe_train!(body)
+
+        dict_frames = drain_pending_dict_frames
+        wire = compress_or_plain(body)
+        [wire, dict_frames]
+      end
+
+
+      # Decodes a wire message. Returns the plaintext String, or `nil`
+      # if this message was a dict frame (installed into the receive
+      # store, not surfaced to the caller).
+      def decode(wire)
+        raise ProtocolError, "wire message too short" if wire.bytesize < 4
+        head = wire.byteslice(0, 4)
+        case head
+        when NUL_PREAMBLE
+          wire.byteslice(4, wire.bytesize - 4) || "".b
+        when ZSTD_MAGIC
+          decode_zstd_frame(wire)
+        when ZDICT_MAGIC
+          install_recv_dict(wire)
+          nil
+        else
+          raise ProtocolError, "unrecognized preamble: #{head.unpack1('H*')}"
+        end
+      end
+
+
+      private
+
+
+      def maybe_train!(body)
+        return unless @training
+        return if body.bytesize >= TRAIN_MAX_SAMPLE_LEN
+
+        @train_samples << body
+        @train_bytes += body.bytesize
+
+        return unless @train_samples.size >= TRAIN_MAX_SAMPLES ||
+                      @train_bytes >= TRAIN_MAX_BYTES
+
+        begin
+          bytes = RZstd::Dictionary.train(@train_samples, capacity: DICT_CAPACITY)
+        rescue RuntimeError
+          @training = false
+          @train_samples = nil
+          return
+        end
+
+        @training = false
+        @train_samples = nil
+
+        patched = patch_auto_dict_id(bytes)
+        install_send_dict(patched)
+      end
+
+
+      # Rewrite bytes [4..7] of a freshly trained dict with a random id
+      # in USER_DICT_ID_RANGE, to avoid reserved ranges per §4.3.
+      def patch_auto_dict_id(bytes)
+        out = bytes.dup.b
+        id  = rand(USER_DICT_ID_RANGE)
+        out[4, 4] = [id].pack("V")
+        out
+      end
+
+
+      def install_send_dict(bytes)
+        if @send_dict_order.size >= MAX_DICTS
+          raise ProtocolError, "send-side dict count would exceed #{MAX_DICTS}"
+        end
+        total = @send_dict_bytes.each_value.sum(&:bytesize) + bytes.bytesize
+        if total > MAX_DICTS_TOTAL_BYTES
+          raise ProtocolError, "send-side dict bytes would exceed #{MAX_DICTS_TOTAL_BYTES}"
+        end
+        unless bytes.byteslice(0, 4) == ZDICT_MAGIC
+          raise ProtocolError, "supplied dict is not ZDICT-format"
+        end
+
+        dict = RZstd::Dictionary.new(bytes, level: @level)
+        id   = dict.id
+        return if @send_dicts.key?(id)
+
+        @send_dicts[id]      = dict
+        @send_dict_bytes[id] = bytes
+        @send_dict_order << id
+        @pending_ship << id
+        @active_send_id ||= id
+      end
+
+
+      def drain_pending_dict_frames
+        return [] if @pending_ship.empty?
+
+        frames = @pending_ship.map { |id| @send_dict_bytes.fetch(id) }
+        @pending_ship = []
+        frames
+      end
+
+
+      def compress_or_plain(body)
+        threshold = @active_send_id ? MIN_COMPRESS_WITH_DICT : MIN_COMPRESS_NO_DICT
+        return plain(body) if body.bytesize < threshold
+
+        compressed =
+          if @active_send_id
+            @send_dicts.fetch(@active_send_id).compress(body)
+          else
+            RZstd.compress(body, level: @level)
+          end
+
+        # Sanity bailout: a compressed result that doesn't save at
+        # least four bytes gets emitted as plaintext instead. Avoids
+        # paying a preamble's worth of overhead for negative wins.
+        return plain(body) if compressed.bytesize >= body.bytesize - 4
+
+        compressed
+      end
+
+
+      def plain(body)
+        NUL_PREAMBLE + body
+      end
+
+
+      def decode_zstd_frame(wire)
+        fcs = parse_frame_content_size(wire)
+        raise ProtocolError, "Zstd frame missing Frame_Content_Size" if fcs.nil?
+        if fcs > @recv_max_size
+          raise ProtocolError, "declared FCS #{fcs} exceeds limit #{@recv_max_size}"
+        end
+
+        dict_id = parse_frame_dict_id(wire)
+        if dict_id && dict_id != 0
+          dict = @recv_dicts[dict_id]
+          raise ProtocolError, "unknown dict_id #{dict_id}" if dict.nil?
+          dict.decompress(wire, max_output_size: @recv_max_size)
+        else
+          RZstd.decompress(wire, max_output_size: @recv_max_size)
+        end
+      rescue RZstd::DecompressError => e
+        raise ProtocolError, "decompression failed: #{e.message}"
+      rescue RZstd::MissingContentSizeError => e
+        raise ProtocolError, "Zstd frame missing Frame_Content_Size (#{e.message})"
+      rescue RZstd::OutputSizeLimitError => e
+        raise ProtocolError, "declared FCS exceeds limit (#{e.message})"
+      end
+
+
+      def install_recv_dict(wire)
+        if @recv_dicts.size >= MAX_DICTS
+          raise ProtocolError, "recv-side dict count would exceed #{MAX_DICTS}"
+        end
+        total = @recv_total_bytes + wire.bytesize
+        if total > MAX_DICTS_TOTAL_BYTES
+          raise ProtocolError, "recv-side dict bytes would exceed #{MAX_DICTS_TOTAL_BYTES}"
+        end
+        if wire.bytesize < 8
+          raise ProtocolError, "dict frame too short"
+        end
+
+        id = wire.byteslice(4, 4).unpack1("V")
+        dict = RZstd::Dictionary.new(wire.b)
+        unless dict.id == id
+          raise ProtocolError, "dict header id mismatch"
+        end
+
+        if (existing = @recv_dicts[id])
+          # Idempotent overwrite: adjust running total.
+          @recv_total_bytes -= @send_dict_bytes[id]&.bytesize || 0
+          _ = existing
+        end
+        @recv_dicts[id] = dict
+        @recv_total_bytes += wire.bytesize
+      end
+
+
+      # Parses the Zstandard `Frame_Content_Size` field from a frame
+      # header. Returns the FCS as an Integer, or `nil` if absent.
+      # Per the Zstandard spec, FCS is absent iff
+      # `Single_Segment_flag == 0 && FCS_flag == 0`.
+      def parse_frame_content_size(wire)
+        return nil if wire.bytesize < 5
+        fhd        = wire.getbyte(4)
+        did_flag   = fhd & 0x03
+        single_seg = (fhd >> 5) & 0x01
+        fcs_flag   = (fhd >> 6) & 0x03
+
+        return nil if fcs_flag == 0 && single_seg == 0
+
+        off = 5 + (single_seg == 0 ? 1 : 0) + [0, 1, 2, 4][did_flag]
+        case fcs_flag
+        when 0
+          return nil if wire.bytesize < off + 1
+          wire.getbyte(off)
+        when 1
+          return nil if wire.bytesize < off + 2
+          wire.byteslice(off, 2).unpack1("v") + 256
+        when 2
+          return nil if wire.bytesize < off + 4
+          wire.byteslice(off, 4).unpack1("V")
+        when 3
+          return nil if wire.bytesize < off + 8
+          lo, hi = wire.byteslice(off, 8).unpack("VV")
+          (hi << 32) | lo
+        end
+      end
+
+
+      # Parses the `Dictionary_ID` field from a Zstd frame header.
+      # Returns the id as an Integer (0 if the frame carries no
+      # Dictionary_ID field), or `nil` if the header is truncated.
+      def parse_frame_dict_id(wire)
+        return nil if wire.bytesize < 5
+        fhd        = wire.getbyte(4)
+        did_flag   = fhd & 0x03
+        single_seg = (fhd >> 5) & 0x01
+
+        off = 5 + (single_seg == 0 ? 1 : 0)
+        case did_flag
+        when 0
+          0
+        when 1
+          return nil if wire.bytesize < off + 1
+          wire.getbyte(off)
+        when 2
+          return nil if wire.bytesize < off + 2
+          wire.byteslice(off, 2).unpack1("v")
+        when 3
+          return nil if wire.bytesize < off + 4
+          wire.byteslice(off, 4).unpack1("V")
+        end
+      end
+    end
+  end
+end

data/lib/nnq/zstd/version.rb

@@ -0,0 +1,7 @@
+# frozen_string_literal: true
+
+module NNQ
+  module Zstd
+    VERSION = "0.1.0"
+  end
+end

data/lib/nnq/zstd/wrapper.rb

@@ -0,0 +1,112 @@
+# frozen_string_literal: true
+
+module NNQ
+  module Zstd
+    # Socket decorator that transparently runs every outbound body
+    # through the Codec's encoder and every inbound wire message
+    # through its decoder. Delegates any unknown method to the wrapped
+    # socket, so the wrapper quacks like an `NNQ::Socket`.
+    class Wrapper
+      attr_reader :codec
+
+
+      def initialize(socket, level:, dicts:)
+        @sock  = socket
+        @codec = Codec.new(
+          level:         level,
+          dicts:         dicts,
+          recv_max_size: recv_max_size_from(socket),
+        )
+        start_dict_monitor!
+      end
+
+
+      def send(body)
+        send_with_codec(body) { |wire| @sock.send(wire) }
+      end
+
+
+      def send_reply(body)
+        send_with_codec(body) { |wire| @sock.send_reply(wire) }
+      end
+
+
+      def send_survey(body)
+        send_with_codec(body) { |wire| @sock.send_survey(wire) }
+      end
+
+
+      def send_request(body)
+        send_with_codec(body) { |wire| @sock.send_request(wire) }
+      end
+
+
+      # Loops internally until a real payload arrives or the socket
+      # closes. Dict frames are silently installed and discarded.
+      def receive
+        loop do
+          raw = @sock.receive
+          return raw if raw.nil?
+          decoded = @codec.decode(raw)
+          return decoded unless decoded.nil?
+        end
+      end
+
+
+      def close
+        begin
+          @monitor_task&.stop
+        rescue StandardError
+          # Monitor task may already be gone; fine.
+        end
+        @sock.close
+      end
+
+
+      def respond_to_missing?(name, include_private = false)
+        @sock.respond_to?(name, include_private)
+      end
+
+
+      def method_missing(name, *args, **kwargs, &block)
+        if @sock.respond_to?(name)
+          @sock.public_send(name, *args, **kwargs, &block)
+        else
+          super
+        end
+      end
+
+
+      private
+
+
+      def send_with_codec(body)
+        wire, dict_frames = @codec.encode(body)
+        dict_frames.each { |df| yield(df) }
+        yield(wire)
+      end
+
+
+      def recv_max_size_from(socket)
+        socket.options.recv_maxsz
+      rescue NoMethodError
+        nil
+      end
+
+
+      # Best-effort: when a new peer connects, requeue every known
+      # dict so the next encode call re-ships them in front of its
+      # payload. Uses the underlying socket's monitor stream.
+      def start_dict_monitor!
+        return unless @sock.respond_to?(:monitor)
+
+        @monitor_task = @sock.monitor do |event|
+          @codec.requeue_all_dicts_for_shipping! if event.type == :connected
+        end
+      rescue StandardError
+        # Monitor unavailable on this socket; dict re-shipping on
+        # reconnect just won't fire. Not fatal.
+      end
+    end
+  end
+end

data/lib/nnq/zstd.rb

@@ -0,0 +1,45 @@
+# frozen_string_literal: true
+
+require "nnq"
+require "rzstd"
+require "set"
+
+require_relative "zstd/version"
+
+module NNQ
+  module Zstd
+    class ProtocolError < StandardError; end
+  end
+end
+
+require_relative "zstd/codec"
+require_relative "zstd/wrapper"
+
+module NNQ
+  module Zstd
+
+    RESERVED_DICT_ID_LOW_MAX  = 32_767
+    RESERVED_DICT_ID_HIGH_MIN = 2**31
+    USER_DICT_ID_RANGE        = (32_768..(2**31 - 1)).freeze
+
+    # Wraps an NNQ::Socket with transparent Zstd compression.
+    #
+    # Both peers must wrap their sockets; there is no negotiation. See
+    # RFC.md for the wire protocol.
+    #
+    # @param socket  [NNQ::Socket]
+    # @param level   [Integer] Zstd level (default -3; fast strategy)
+    # @param dict    [String, Array<String>, nil] pre-built dictionary
+    #   bytes. If provided, training is skipped and all supplied dicts
+    #   are shipped to peers on the wire. Each buffer MUST be a valid
+    #   Zstd dictionary (ZDICT magic + header).
+    #
+    # A receive-only "passive" decoder needs no special flag — wrap
+    # the socket and just never call `send`. Training is driven by
+    # outbound traffic, so a socket that only receives naturally
+    # skips training and dict shipping.
+    def self.wrap(socket, level: -3, dict: nil)
+      Wrapper.new(socket, level: level, dicts: Array(dict).compact)
+    end
+  end
+end

metadata

@@ -0,0 +1,78 @@
+--- !ruby/object:Gem::Specification
+name: nnq-zstd
+version: !ruby/object:Gem::Version
+  version: 0.1.0
+platform: ruby
+authors:
+- Patrik Wenger
+bindir: bin
+cert_chain: []
+date: 1980-01-02 00:00:00.000000000 Z
+dependencies:
+- !ruby/object:Gem::Dependency
+  name: nnq
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '0.5'
+  type: :runtime
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '0.5'
+- !ruby/object:Gem::Dependency
+  name: rzstd
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '0.3'
+  type: :runtime
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '0.3'
+description: Wraps any NNQ::Socket with per-message Zstd compression, bounded decompression,
+  in-band dictionary shipping, and sender-side dictionary training. No negotiation;
+  both peers must wrap.
+email:
+- paddor@gmail.com
+executables: []
+extensions: []
+extra_rdoc_files: []
+files:
+- CHANGELOG.md
+- LICENSE
+- README.md
+- RFC.md
+- lib/nnq/zstd.rb
+- lib/nnq/zstd/codec.rb
+- lib/nnq/zstd/version.rb
+- lib/nnq/zstd/wrapper.rb
+homepage: https://github.com/paddor/nnq-zstd
+licenses:
+- ISC
+metadata: {}
+rdoc_options: []
+require_paths:
+- lib
+required_ruby_version: !ruby/object:Gem::Requirement
+  requirements:
+  - - ">="
+    - !ruby/object:Gem::Version
+      version: '4.0'
+required_rubygems_version: !ruby/object:Gem::Requirement
+  requirements:
+  - - ">="
+    - !ruby/object:Gem::Version
+      version: '0'
+requirements: []
+rubygems_version: 4.0.6
+specification_version: 4
+summary: Transparent Zstd compression wrapper for NNQ sockets
+test_files: []