dalli 4.1.0 → 4.3.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: e342d39d58d552607783486a9c9f0ffa23d9c479079c62cb760ec84a97d064da
-  data.tar.gz: 1ae923ede204d0a3e82426404cfb4882f4414a12cb351cf3d04b9ef8653051ce
+  metadata.gz: e97e2a407956737b411c33627d1f771be758017b881c68fab66edee95dc3249e
+  data.tar.gz: b57638f133a592e9d57b71530bc925ff6589b27549e8785bc6c7f334906636ff
 SHA512:
-  metadata.gz: aeb1bec84092f4e07db884b415d6b25375dc995fff04cc58a1764d9ad0937986ba7ff5db5a665a831bd8ab488b3fa4868fce79834e93a26848bc37ac1c9f9855
-  data.tar.gz: 8c5a4e21f4b0a86279bf28edc729c0a1981964438342117f3bc180d15d5cdfc93e395913267db7af66b3907416051be7629e687708c163f4d4015d4e2a768508
+  metadata.gz: 7dd43e9e5b09b65174f2e46b84de40e4e5bcfae9645c6bb67017ca81a64d344e7345cc4bf685f11d4779aeaafd7c7163df5a67bf8410153034f4adf523385b95
+  data.tar.gz: b57506702dbdcb387490b3c0c56d2407aff4f0f789aec01e2b1a92c9f8d925a0225b8563e3dbc1ccf3c3ca72a25ca58f4199f1738d93bdf65a07893ed2930b6d

data/CHANGELOG.md

@@ -1,6 +1,56 @@
 Dalli Changelog
 =====================
 
+4.3.0
+==========
+
+New Features:
+
+- Add `namespace_separator` option to customize the separator between namespace and key (#1019)
+  - Default is `:` for backward compatibility
+  - Must be a single non-alphanumeric character (e.g., `:`, `/`, `|`, `.`)
+  - Example: `Dalli::Client.new(servers, namespace: 'myapp', namespace_separator: '/')`
+
+Bug Fixes:
+
+- Fix architecture-dependent struct timeval packing for socket timeouts (#1034)
+  - Detects correct pack format for time_t and suseconds_t on each platform
+  - Fixes timeout issues on architectures with 64-bit time_t
+
+- Fix get_multi hanging with large key counts (#776, #941)
+  - Add interleaved read/write for pipelined gets to prevent socket buffer deadlock
+  - For batches over 10,000 keys per server, requests are now sent in chunks
+
+- **Breaking:** Enforce string-only values in raw mode (#1022)
+  - `set(key, nil, raw: true)` now raises `MarshalError` instead of storing `""`
+  - `set(key, 123, raw: true)` now raises `MarshalError` instead of storing `"123"`
+  - This matches the behavior of client-level `raw: true` mode
+  - To store counters, use string values: `set('counter', '0', raw: true)`
+
+CI:
+
+- Add TruffleRuby to CI test matrix (#988)
+
+4.2.0
+==========
+
+Performance:
+
+- Buffered I/O: Use `socket.sync = false` with explicit flush to reduce syscalls for pipelined operations
+- get_multi optimizations: Use Set for O(1) server tracking lookups
+- Raw mode optimization: Skip bitflags request in meta protocol when in raw mode (saves 2 bytes per request)
+
+New Features:
+
+- OpenTelemetry tracing support: Automatically instruments operations when OpenTelemetry SDK is present
+  - Zero overhead when OpenTelemetry is not loaded
+  - Traces `get`, `set`, `delete`, `get_multi`, `set_multi`, `delete_multi`, `get_with_metadata`, and `fetch_with_lock`
+  - Spans include `db.system: memcached` and `db.operation` attributes
+  - Single-key operations include `server.address` attribute
+  - Multi-key operations include `db.memcached.key_count` attribute
+  - `get_multi` spans include `db.memcached.hit_count` and `db.memcached.miss_count` for cache efficiency metrics
+  - Exceptions are automatically recorded on spans with error status
+
 4.1.0
 ==========
 

data/README.md

@@ -11,6 +11,7 @@ Dalli supports:
 * Thread-safe operation (either through use of a connection pool, or by using the Dalli client in threadsafe mode)
 * SSL/TLS connections to memcached
 * SASL authentication
+* OpenTelemetry distributed tracing (automatic when SDK is present)
 
 The name is a variant of Salvador Dali for his famous painting [The Persistence of Memory](http://en.wikipedia.org/wiki/The_Persistence_of_Memory).
 
@@ -30,6 +31,31 @@ Dalli supports two protocols for communicating with memcached:
 Dalli::Client.new('localhost:11211', protocol: :meta)
 ```
 
+## Configuration Options
+
+### Namespace
+
+Use namespaces to partition your cache and avoid key collisions between different applications or environments:
+
+```ruby
+# All keys will be prefixed with "myapp:"
+Dalli::Client.new('localhost:11211', namespace: 'myapp')
+
+# Dynamic namespace using a Proc (evaluated on each operation)
+Dalli::Client.new('localhost:11211', namespace: -> { "tenant:#{Thread.current[:tenant_id]}" })
+```
+
+### Namespace Separator
+
+By default, the namespace and key are joined with a colon (`:`). You can customize this with the `namespace_separator` option:
+
+```ruby
+# Keys will be prefixed with "myapp/" instead of "myapp:"
+Dalli::Client.new('localhost:11211', namespace: 'myapp', namespace_separator: '/')
+```
+
+The separator must be a single non-alphanumeric character. Valid examples: `:`, `/`, `|`, `.`, `-`, `_`, `#`
+
 ## Security Note
 
 By default, Dalli uses Ruby's Marshal for serialization. Deserializing untrusted data with Marshal can lead to remote code execution. If you cache user-controlled data, consider using a safer serializer:
@@ -40,6 +66,46 @@ Dalli::Client.new('localhost:11211', serializer: JSON)
 
 See the [4.0-Upgrade.md](4.0-Upgrade.md) guide for more information.
 
+## OpenTelemetry Tracing
+
+Dalli automatically instruments operations with [OpenTelemetry](https://opentelemetry.io/) when the SDK is present. No configuration is required - just add the OpenTelemetry gems to your application:
+
+```ruby
+# Gemfile
+gem 'opentelemetry-sdk'
+gem 'opentelemetry-exporter-otlp' # or your preferred exporter
+```
+
+When OpenTelemetry is loaded, Dalli creates spans for:
+- Single key operations: `get`, `set`, `delete`, `add`, `replace`, `incr`, `decr`, etc.
+- Multi-key operations: `get_multi`, `set_multi`, `delete_multi`
+- Advanced operations: `get_with_metadata`, `fetch_with_lock`
+
+### Span Attributes
+
+All spans include:
+- `db.system`: `memcached`
+- `db.operation`: The operation name (e.g., `get`, `set_multi`)
+
+Single-key operations also include:
+- `server.address`: The memcached server that handled the request (e.g., `localhost:11211`)
+
+Multi-key operations include cache efficiency metrics:
+- `db.memcached.key_count`: Number of keys in the request
+- `db.memcached.hit_count`: Number of keys found (for `get_multi`)
+- `db.memcached.miss_count`: Number of keys not found (for `get_multi`)
+
+### Error Handling
+
+Exceptions are automatically recorded on spans with error status. When an operation fails:
+1. The exception is recorded on the span via `span.record_exception(e)`
+2. The span status is set to error with the exception message
+3. The exception is re-raised to the caller
+
+### Zero Overhead
+
+When OpenTelemetry is not present, there is zero overhead - the tracing code checks once at startup and bypasses all instrumentation logic entirely when the SDK is not loaded.
+
 ![Persistence of Memory](https://upload.wikimedia.org/wikipedia/en/d/dd/The_Persistence_of_Memory.jpg)
 
 

data/lib/dalli/client.rb

@@ -134,8 +134,10 @@ module Dalli
       key = key.to_s
       key = @key_manager.validate_key(key)
 
-      server = ring.server_for_key(key)
-      server.request(:meta_get, key, options)
+      Instrumentation.trace('get_with_metadata', { 'db.operation' => 'get_with_metadata' }) do
+        server = ring.server_for_key(key)
+        server.request(:meta_get, key, options)
+      end
     rescue NetworkError => e
       Dalli.logger.debug { e.inspect }
       Dalli.logger.debug { 'retrying get_with_metadata with new server' }
@@ -146,20 +148,19 @@ module Dalli
     # Fetch multiple keys efficiently.
     # If a block is given, yields key/value pairs one at a time.
     # Otherwise returns a hash of { 'key' => 'value', 'key2' => 'value1' }
+    # rubocop:disable Style/ExplicitBlockArgument
     def get_multi(*keys)
       keys.flatten!
       keys.compact!
-
       return {} if keys.empty?
 
       if block_given?
-        pipelined_getter.process(keys) { |k, data| yield k, data.first }
+        get_multi_yielding(keys) { |k, v| yield k, v }
       else
-        {}.tap do |hash|
-          pipelined_getter.process(keys) { |k, data| hash[k] = data.first }
-        end
+        get_multi_hash(keys)
       end
     end
+    # rubocop:enable Style/ExplicitBlockArgument
 
     ##
     # Fetch multiple keys efficiently, including available metadata such as CAS.
@@ -228,7 +229,7 @@ module Dalli
     #     expensive_operation
     #   end
     #
-    def fetch_with_lock(key, ttl: nil, lock_ttl: 30, recache_threshold: nil, req_options: nil)
+    def fetch_with_lock(key, ttl: nil, lock_ttl: 30, recache_threshold: nil, req_options: nil, &block)
       raise ArgumentError, 'Block is required for fetch_with_lock' unless block_given?
 
       raise_unless_meta_protocol!
@@ -236,21 +237,8 @@ module Dalli
       key = key.to_s
       key = @key_manager.validate_key(key)
 
-      server = ring.server_for_key(key)
-      result = server.request(:meta_get, key, {
-                                vivify_ttl: lock_ttl,
-                                recache_ttl: recache_threshold
-                              })
-
-      if result[:won_recache]
-        # This client won the race - regenerate the value
-        new_val = yield
-        set(key, new_val, ttl_or_default(ttl), req_options)
-        new_val
-      else
-        # Another client is regenerating, or value exists and isn't stale
-        # Return the existing value
-        result[:value]
+      Instrumentation.trace('fetch_with_lock', { 'db.operation' => 'fetch_with_lock' }) do
+        fetch_with_lock_request(key, ttl, lock_ttl, recache_threshold, req_options, &block)
       end
     rescue NetworkError => e
       Dalli.logger.debug { e.inspect }
@@ -330,7 +318,12 @@ module Dalli
     def set_multi(hash, ttl = nil, req_options = nil)
       return if hash.empty?
 
-      pipelined_setter.process(hash, ttl_or_default(ttl), req_options)
+      Instrumentation.trace('set_multi', {
+                              'db.operation' => 'set_multi',
+                              'db.memcached.key_count' => hash.size
+                            }) do
+        pipelined_setter.process(hash, ttl_or_default(ttl), req_options)
+      end
     end
 
     ##
@@ -385,7 +378,12 @@ module Dalli
     def delete_multi(keys)
       return if keys.empty?
 
-      pipelined_deleter.process(keys)
+      Instrumentation.trace('delete_multi', {
+                              'db.operation' => 'delete_multi',
+                              'db.memcached.key_count' => keys.size
+                            }) do
+        pipelined_deleter.process(keys)
+      end
     end
 
     ##
@@ -517,6 +515,42 @@ module Dalli
 
     private
 
+    # Records hit/miss metrics on a span for cache observability.
+    # @param span [OpenTelemetry::Trace::Span, nil] the span to record on
+    # @param key_count [Integer] total keys requested
+    # @param hit_count [Integer] keys found in cache
+    def record_hit_miss_metrics(span, key_count, hit_count)
+      return unless span
+
+      span.set_attribute('db.memcached.hit_count', hit_count)
+      span.set_attribute('db.memcached.miss_count', key_count - hit_count)
+    end
+
+    def get_multi_yielding(keys)
+      Instrumentation.trace_with_result('get_multi', get_multi_attributes(keys)) do |span|
+        hit_count = 0
+        pipelined_getter.process(keys) do |k, data|
+          hit_count += 1
+          yield k, data.first
+        end
+        record_hit_miss_metrics(span, keys.size, hit_count)
+        nil
+      end
+    end
+
+    def get_multi_hash(keys)
+      Instrumentation.trace_with_result('get_multi', get_multi_attributes(keys)) do |span|
+        {}.tap do |hash|
+          pipelined_getter.process(keys) { |k, data| hash[k] = data.first }
+          record_hit_miss_metrics(span, keys.size, hash.size)
+        end
+      end
+    end
+
+    def get_multi_attributes(keys)
+      { 'db.operation' => 'get_multi', 'db.memcached.key_count' => keys.size }
+    end
+
     def check_positive!(amt)
       raise ArgumentError, "Positive values only: #{amt}" if amt.negative?
     end
@@ -529,6 +563,17 @@ module Dalli
       perform(:set, key, newvalue, ttl_or_default(ttl), cas, req_options)
     end
 
+    def fetch_with_lock_request(key, ttl, lock_ttl, recache_threshold, req_options)
+      server = ring.server_for_key(key)
+      result = server.request(:meta_get, key, { vivify_ttl: lock_ttl, recache_ttl: recache_threshold })
+
+      return result[:value] unless result[:won_recache]
+
+      new_val = yield
+      set(key, new_val, ttl_or_default(ttl), req_options)
+      new_val
+    end
+
     ##
     # Uses the argument TTL or the client-wide default.  Ensures
     # that the value is an integer
@@ -571,7 +616,12 @@ module Dalli
       key = @key_manager.validate_key(key)
 
       server = ring.server_for_key(key)
-      server.request(op, key, *args)
+      Instrumentation.trace(op.to_s, {
+                              'db.operation' => op.to_s,
+                              'server.address' => server.name
+                            }) do
+        server.request(op, key, *args)
+      end
     rescue NetworkError => e
       Dalli.logger.debug { e.inspect }
       Dalli.logger.debug { 'retrying request with new server' }

data/lib/dalli/instrumentation.rb

@@ -0,0 +1,139 @@
+# frozen_string_literal: true
+
+module Dalli
+  ##
+  # Instrumentation support for Dalli. Provides hooks for distributed tracing
+  # via OpenTelemetry when the SDK is available.
+  #
+  # When OpenTelemetry is loaded, Dalli automatically creates spans for cache operations.
+  # When OpenTelemetry is not available, all tracing methods are no-ops with zero overhead.
+  #
+  # == Span Attributes
+  #
+  # All spans include the following default attributes:
+  # - +db.system+ - Always "memcached"
+  #
+  # Single-key operations (+get+, +set+, +delete+, +incr+, +decr+, etc.) add:
+  # - +db.operation+ - The operation name (e.g., "get", "set")
+  # - +server.address+ - The memcached server handling the request (e.g., "localhost:11211")
+  #
+  # Multi-key operations (+get_multi+) add:
+  # - +db.operation+ - "get_multi"
+  # - +db.memcached.key_count+ - Number of keys requested
+  # - +db.memcached.hit_count+ - Number of keys found in cache
+  # - +db.memcached.miss_count+ - Number of keys not found
+  #
+  # Bulk write operations (+set_multi+, +delete_multi+) add:
+  # - +db.operation+ - The operation name
+  # - +db.memcached.key_count+ - Number of keys in the operation
+  #
+  # == Error Handling
+  #
+  # When an exception occurs during a traced operation:
+  # - The exception is recorded on the span via +record_exception+
+  # - The span status is set to error with the exception message
+  # - The exception is re-raised to the caller
+  #
+  # @example Checking if tracing is enabled
+  #   Dalli::Instrumentation.enabled? # => true if OpenTelemetry is loaded
+  #
+  ##
+  module Instrumentation
+    # Default attributes included on all memcached spans.
+    # @return [Hash] frozen hash with 'db.system' => 'memcached'
+    DEFAULT_ATTRIBUTES = { 'db.system' => 'memcached' }.freeze
+
+    class << self
+      # Returns the OpenTelemetry tracer if available, nil otherwise.
+      #
+      # The tracer is cached after first lookup for performance.
+      # Uses the library name 'dalli' and current Dalli::VERSION.
+      #
+      # @return [OpenTelemetry::Trace::Tracer, nil] the tracer or nil if OTel unavailable
+      def tracer
+        return @tracer if defined?(@tracer)
+
+        @tracer = (OpenTelemetry.tracer_provider.tracer('dalli', Dalli::VERSION) if defined?(OpenTelemetry))
+      end
+
+      # Returns true if instrumentation is enabled (OpenTelemetry SDK is available).
+      #
+      # @return [Boolean] true if tracing is active, false otherwise
+      def enabled?
+        !tracer.nil?
+      end
+
+      # Wraps a block with a span if instrumentation is enabled.
+      #
+      # Creates a client span with the given name and attributes merged with
+      # DEFAULT_ATTRIBUTES. The block is executed within the span context.
+      # If an exception occurs, it is recorded on the span before re-raising.
+      #
+      # When tracing is disabled (OpenTelemetry not loaded), this method
+      # simply yields directly with zero overhead.
+      #
+      # @param name [String] the span name (e.g., 'get', 'set', 'delete')
+      # @param attributes [Hash] span attributes to merge with defaults.
+      #   Common attributes include:
+      #   - 'db.operation' - the operation name
+      #   - 'server.address' - the target server
+      #   - 'db.memcached.key_count' - number of keys (for multi operations)
+      # @yield the cache operation to trace
+      # @return [Object] the result of the block
+      # @raise [StandardError] re-raises any exception from the block
+      #
+      # @example Tracing a set operation
+      #   trace('set', { 'db.operation' => 'set', 'server.address' => 'localhost:11211' }) do
+      #     server.set(key, value, ttl)
+      #   end
+      #
+      def trace(name, attributes = {})
+        return yield unless enabled?
+
+        tracer.in_span(name, attributes: DEFAULT_ATTRIBUTES.merge(attributes), kind: :client) do |span|
+          yield
+        rescue StandardError => e
+          span.record_exception(e)
+          span.status = OpenTelemetry::Trace::Status.error(e.message)
+          raise
+        end
+      end
+
+      # Like trace, but yields the span to allow adding attributes after execution.
+      #
+      # This is useful for operations where metrics are only known after the
+      # operation completes, such as get_multi where hit/miss counts depend
+      # on the cache response.
+      #
+      # When tracing is disabled, yields nil as the span argument.
+      #
+      # @param name [String] the span name (e.g., 'get_multi')
+      # @param attributes [Hash] initial span attributes to merge with defaults
+      # @yield [OpenTelemetry::Trace::Span, nil] the span object, or nil if disabled
+      # @return [Object] the result of the block
+      # @raise [StandardError] re-raises any exception from the block
+      #
+      # @example Recording hit/miss metrics after get_multi
+      #   trace_with_result('get_multi', { 'db.operation' => 'get_multi' }) do |span|
+      #     results = fetch_from_cache(keys)
+      #     if span
+      #       span.set_attribute('db.memcached.hit_count', results.size)
+      #       span.set_attribute('db.memcached.miss_count', keys.size - results.size)
+      #     end
+      #     results
+      #   end
+      #
+      def trace_with_result(name, attributes = {})
+        return yield(nil) unless enabled?
+
+        tracer.in_span(name, attributes: DEFAULT_ATTRIBUTES.merge(attributes), kind: :client) do |span|
+          yield(span)
+        rescue StandardError => e
+          span.record_exception(e)
+          span.status = OpenTelemetry::Trace::Status.error(e.message)
+          raise
+        end
+      end
+    end
+  end
+end

data/lib/dalli/key_manager.rb

@@ -12,7 +12,7 @@ module Dalli
   class KeyManager
     MAX_KEY_LENGTH = 250
 
-    NAMESPACE_SEPARATOR = ':'
+    DEFAULT_NAMESPACE_SEPARATOR = ':'
 
     # This is a hard coded md5 for historical reasons
     TRUNCATED_KEY_SEPARATOR = ':md5:'
@@ -21,19 +21,26 @@ module Dalli
     TRUNCATED_KEY_TARGET_SIZE = 249
 
     DEFAULTS = {
-      digest_class: ::Digest::MD5
+      digest_class: ::Digest::MD5,
+      namespace_separator: DEFAULT_NAMESPACE_SEPARATOR
     }.freeze
 
-    OPTIONS = %i[digest_class namespace].freeze
+    OPTIONS = %i[digest_class namespace namespace_separator].freeze
 
-    attr_reader :namespace
+    attr_reader :namespace, :namespace_separator
+
+    # Valid separators: non-alphanumeric, single printable ASCII characters
+    # Excludes: alphanumerics, whitespace, control characters
+    VALID_NAMESPACE_SEPARATORS = /\A[^a-zA-Z0-9\s\x00-\x1F\x7F]\z/
 
     def initialize(client_options)
       @key_options =
         DEFAULTS.merge(client_options.slice(*OPTIONS))
       validate_digest_class_option(@key_options)
+      validate_namespace_separator_option(@key_options)
 
       @namespace = namespace_from_options
+      @namespace_separator = @key_options[:namespace_separator]
     end
 
     ##
@@ -61,7 +68,7 @@ module Dalli
     def key_with_namespace(key)
       return key if namespace.nil?
 
-      "#{evaluate_namespace}#{NAMESPACE_SEPARATOR}#{key}"
+      "#{evaluate_namespace}#{namespace_separator}#{key}"
     end
 
     def key_without_namespace(key)
@@ -75,9 +82,9 @@ module Dalli
     end
 
     def namespace_regexp
-      return /\A#{Regexp.escape(evaluate_namespace)}:/ if namespace.is_a?(Proc)
+      return /\A#{Regexp.escape(evaluate_namespace)}#{Regexp.escape(namespace_separator)}/ if namespace.is_a?(Proc)
 
-      @namespace_regexp ||= /\A#{Regexp.escape(namespace)}:/ unless namespace.nil?
+      @namespace_regexp ||= /\A#{Regexp.escape(namespace)}#{Regexp.escape(namespace_separator)}/ unless namespace.nil?
     end
 
     def validate_digest_class_option(opts)
@@ -86,6 +93,14 @@ module Dalli
       raise ArgumentError, 'The digest_class object must respond to the hexdigest method'
     end
 
+    def validate_namespace_separator_option(opts)
+      sep = opts[:namespace_separator]
+      return if VALID_NAMESPACE_SEPARATORS.match?(sep)
+
+      raise ArgumentError,
+            'namespace_separator must be a single non-alphanumeric character (e.g., ":", "/", "|")'
+    end
+
     def namespace_from_options
       raw_namespace = @key_options[:namespace]
       return nil unless raw_namespace

data/lib/dalli/pipelined_getter.rb

@@ -1,10 +1,19 @@
 # frozen_string_literal: true
 
+require 'set'
+
 module Dalli
   ##
   # Contains logic for the pipelined gets implemented by the client.
   ##
   class PipelinedGetter
+    # For large batches, interleave sends with response draining to prevent
+    # socket buffer deadlock. Only kicks in above this threshold.
+    INTERLEAVE_THRESHOLD = 10_000
+
+    # Number of keys to send before draining responses during interleaved mode
+    CHUNK_SIZE = 10_000
+
     def initialize(ring, key_manager)
       @ring = ring
       @key_manager = key_manager
@@ -17,8 +26,14 @@ module Dalli
       return {} if keys.empty?
 
       @ring.lock do
+        # Stores partial results collected during interleaved send phase
+        @partial_results = {}
         servers = setup_requests(keys)
         start_time = Time.now
+
+        # First yield any partial results collected during interleaved send
+        yield_partial_results(&block)
+
         servers = fetch_responses(servers, start_time, @ring.socket_timeout, &block) until servers.empty?
       end
     rescue NetworkError => e
@@ -27,6 +42,15 @@ module Dalli
       retry
     end
 
+    private
+
+    def yield_partial_results
+      @partial_results.each_pair do |key, value_list|
+        yield @key_manager.key_without_namespace(key), value_list
+      end
+      @partial_results.clear
+    end
+
     def setup_requests(keys)
       groups = groups_for_keys(keys)
       make_getkq_requests(groups)
@@ -45,7 +69,14 @@ module Dalli
     ##
     def make_getkq_requests(groups)
       groups.each do |server, keys_for_server|
-        server.request(:pipelined_get, keys_for_server)
+        if keys_for_server.size <= INTERLEAVE_THRESHOLD
+          # Small batch - send all at once (existing behavior)
+          server.request(:pipelined_get, keys_for_server)
+        else
+          # Large batch - interleave sends with response draining
+          # Pass @partial_results directly to avoid hash allocation/merge overhead
+          server.request(:pipelined_get_interleaved, keys_for_server, CHUNK_SIZE, @partial_results)
+        end
       rescue DalliError, NetworkError => e
         Dalli.logger.debug { e.inspect }
         Dalli.logger.debug { "unable to get keys for server #{server.name}" }
@@ -57,7 +88,7 @@ module Dalli
     # our set, sending the noop to terminate the set of queries.
     ##
     def finish_queries(servers)
-      deleted = []
+      deleted = Set.new
 
       servers.each do |server|
         next unless server.connected?
@@ -67,7 +98,7 @@ module Dalli
         rescue Dalli::NetworkError
           raise
         rescue Dalli::DalliError
-          deleted.append(server)
+          deleted << server
         end
       end
 
@@ -94,7 +125,7 @@ module Dalli
 
     def fetch_responses(servers, start_time, timeout, &block)
       # Remove any servers which are not connected
-      servers.delete_if { |s| !s.connected? }
+      servers.select!(&:connected?)
       return [] if servers.empty?
 
       time_left = remaining_time(start_time, timeout)

data/lib/dalli/protocol/base.rb

@@ -23,10 +23,17 @@ module Dalli
       def initialize(attribs, client_options = {})
         hostname, port, socket_type, @weight, user_creds = ServerConfigParser.parse(attribs)
         @options = client_options.merge(user_creds)
-        @value_marshaller = client_options[:raw] ? StringMarshaller.new(@options) : ValueMarshaller.new(@options)
+        @raw_mode = client_options[:raw]
+        @value_marshaller = @raw_mode ? StringMarshaller.new(@options) : ValueMarshaller.new(@options)
         @connection_manager = ConnectionManager.new(hostname, port, socket_type, @options)
       end
 
+      # Returns true if client is in raw mode (no serialization/compression).
+      # In raw mode, we can skip requesting bitflags from the server.
+      def raw_mode?
+        @raw_mode
+      end
+
       # Chokepoint method for error handling and ensuring liveness
       def request(opkey, *args)
         verify_state(opkey)
@@ -35,8 +42,8 @@ module Dalli
           @connection_manager.start_request!
           response = send(opkey, *args)
 
-          # pipelined_get emit query but doesn't read the response(s)
-          @connection_manager.finish_request! unless opkey == :pipelined_get
+          # pipelined_get/pipelined_get_interleaved emit query but don't read the response(s)
+          @connection_manager.finish_request! unless %i[pipelined_get pipelined_get_interleaved].include?(opkey)
 
           response
         rescue Dalli::MarshalError => e
@@ -74,7 +81,9 @@ module Dalli
       def pipeline_response_setup
         verify_pipelined_state(:getkq)
         write_noop
-        response_buffer.reset
+        # Use ensure_ready instead of reset to preserve any data already buffered
+        # during interleaved pipelined get draining
+        response_buffer.ensure_ready
       end
 
       # Attempt to receive and parse as many key/value pairs as possible
@@ -213,6 +222,11 @@ module Dalli
       end
 
       def pipelined_get(keys)
+        # Clear buffer to remove any stale data from interrupted operations.
+        # Use clear (not reset) to keep pipeline_complete? = true, which is
+        # the expected state before pipeline_response_setup is called.
+        response_buffer.clear
+
         req = +''
         keys.each do |key|
           req << quiet_get_request(key)
@@ -221,6 +235,51 @@ module Dalli
         write(req)
       end
 
+      # For large batches, interleave writing requests with draining responses.
+      # This prevents socket buffer deadlock when sending many keys.
+      # Populates the provided results hash with any responses drained during send.
+      def pipelined_get_interleaved(keys, chunk_size, results)
+        # Initialize the response buffer for draining during send phase
+        response_buffer.ensure_ready
+
+        keys.each_slice(chunk_size) do |chunk|
+          # Build and write this chunk of requests
+          req = +''
+          chunk.each do |key|
+            req << quiet_get_request(key)
+          end
+          write(req)
+          @connection_manager.flush
+
+          # Drain any available responses directly into results hash
+          drain_pipeline_responses(results)
+        end
+      end
+
+      # Non-blocking read and processing of any available pipeline responses.
+      # Used during interleaved pipelined gets to prevent buffer deadlock.
+      # Populates the provided results hash directly to avoid allocation overhead.
+      def drain_pipeline_responses(results)
+        return unless connected?
+
+        # Non-blocking check if socket has data available
+        return unless sock.wait_readable(0)
+
+        # Read available data without blocking
+        response_buffer.read
+
+        # Process any complete responses in the buffer
+        loop do
+          status, cas, key, value = response_buffer.process_single_getk_response
+          break if status.nil? # No complete response available
+
+          results[key] = [value, cas] unless key.nil?
+        end
+      rescue SystemCallError, Dalli::NetworkError
+        # Ignore errors during drain - they'll be handled in fetch_responses
+        nil
+      end
+
       def response_buffer
         @response_buffer ||= ResponseBuffer.new(@connection_manager, response_processor)
       end

data/lib/dalli/protocol/binary.rb

@@ -22,6 +22,7 @@ module Dalli
       def get(key, options = nil)
         req = RequestFormatter.standard_request(opkey: :get, key: key)
         write(req)
+        @connection_manager.flush
         response_processor.get(cache_nils: cache_nils?(options))
       end
 
@@ -33,12 +34,14 @@ module Dalli
         ttl = TtlSanitizer.sanitize(ttl)
         req = RequestFormatter.standard_request(opkey: :gat, key: key, ttl: ttl)
         write(req)
+        @connection_manager.flush
         response_processor.get(cache_nils: cache_nils?(options))
       end
 
       def touch(key, ttl)
         ttl = TtlSanitizer.sanitize(ttl)
         write(RequestFormatter.standard_request(opkey: :touch, key: key, ttl: ttl))
+        @connection_manager.flush
         response_processor.generic_response
       end
 
@@ -47,6 +50,7 @@ module Dalli
       def cas(key)
         req = RequestFormatter.standard_request(opkey: :get, key: key)
         write(req)
+        @connection_manager.flush
         response_processor.data_cas_response
       end
 
@@ -81,6 +85,7 @@ module Dalli
                                                 value: value, bitflags: bitflags,
                                                 ttl: ttl, cas: cas)
         write(req)
+        @connection_manager.flush unless quiet?
         response_processor.storage_response unless quiet?
       end
       # rubocop:enable Metrics/ParameterLists
@@ -97,6 +102,7 @@ module Dalli
 
       def write_append_prepend(opkey, key, value)
         write(RequestFormatter.standard_request(opkey: opkey, key: key, value: value))
+        @connection_manager.flush unless quiet?
         response_processor.no_body_response unless quiet?
       end
 
@@ -105,6 +111,7 @@ module Dalli
         opkey = quiet? ? :deleteq : :delete
         req = RequestFormatter.standard_request(opkey: opkey, key: key, cas: cas)
         write(req)
+        @connection_manager.flush unless quiet?
         response_processor.delete unless quiet?
       end
 
@@ -139,6 +146,7 @@ module Dalli
         initial ||= 0
         write(RequestFormatter.decr_incr_request(opkey: opkey, key: key,
                                                  count: count, initial: initial, expiry: expiry))
+        @connection_manager.flush unless quiet?
         response_processor.decr_incr unless quiet?
       end
 
@@ -146,6 +154,7 @@ module Dalli
       def flush(ttl = 0)
         opkey = quiet? ? :flushq : :flush
         write(RequestFormatter.standard_request(opkey: opkey, ttl: ttl))
+        @connection_manager.flush unless quiet?
         response_processor.no_body_response unless quiet?
       end
 
@@ -159,22 +168,26 @@ module Dalli
       def stats(info = '')
         req = RequestFormatter.standard_request(opkey: :stat, key: info)
         write(req)
+        @connection_manager.flush
         response_processor.stats
       end
 
       def reset_stats
         write(RequestFormatter.standard_request(opkey: :stat, key: 'reset'))
+        @connection_manager.flush
         response_processor.reset
       end
 
       def version
         write(RequestFormatter.standard_request(opkey: :version))
+        @connection_manager.flush
         response_processor.version
       end
 
       def write_noop
         req = RequestFormatter.standard_request(opkey: :noop)
         write(req)
+        @connection_manager.flush
       end
 
       require_relative 'binary/request_formatter'

data/lib/dalli/protocol/connection_manager.rb

@@ -53,6 +53,7 @@ module Dalli
         Dalli.logger.debug { "Dalli::Server#connect #{name}" }
 
         @sock = memcached_socket
+        @sock.sync = false # Enable buffered I/O for better performance
         @pid = PIDCache.pid
         @request_in_progress = false
       rescue SystemCallError, *TIMEOUT_ERRORS, EOFError, SocketError => e
@@ -166,6 +167,12 @@ module Dalli
         error_on_request!(e)
       end
 
+      def flush
+        @sock.flush
+      rescue SystemCallError, *TIMEOUT_ERRORS, *SSL_ERRORS => e
+        error_on_request!(e)
+      end
+
       # Non-blocking read.  Here to support the operation
       # of the get_multi operation
       def read_nonblock

data/lib/dalli/protocol/meta/request_formatter.rb

@@ -37,9 +37,12 @@ module Dalli
         # - l<N>: Seconds since last access
         def self.meta_get(key:, value: true, return_cas: false, ttl: nil, base64: false, quiet: false,
                           vivify_ttl: nil, recache_ttl: nil,
-                          return_hit_status: false, return_last_access: false, skip_lru_bump: false)
+                          return_hit_status: false, return_last_access: false, skip_lru_bump: false,
+                          skip_flags: false)
           cmd = "mg #{key}"
-          cmd << ' v f' if value
+          # In raw mode (skip_flags: true), we don't request bitflags since they're not used.
+          # This saves 2 bytes per request and skips parsing on response.
+          cmd << (skip_flags ? ' v' : ' v f') if value
           cmd << ' c' if return_cas
           cmd << ' b' if base64
           cmd << " T#{ttl}" if ttl

data/lib/dalli/protocol/meta.rb

@@ -25,21 +25,28 @@ module Dalli
       # Retrieval Commands
       def get(key, options = nil)
         encoded_key, base64 = KeyRegularizer.encode(key)
-        req = RequestFormatter.meta_get(key: encoded_key, base64: base64)
+        # Skip bitflags in raw mode - saves 2 bytes per request and skips parsing
+        skip_flags = raw_mode? || (options && options[:raw])
+        req = RequestFormatter.meta_get(key: encoded_key, base64: base64, skip_flags: skip_flags)
         write(req)
+        @connection_manager.flush
         response_processor.meta_get_with_value(cache_nils: cache_nils?(options))
       end
 
       def quiet_get_request(key)
         encoded_key, base64 = KeyRegularizer.encode(key)
-        RequestFormatter.meta_get(key: encoded_key, return_cas: true, base64: base64, quiet: true)
+        # Skip bitflags in raw mode - saves 2 bytes per request and skips parsing
+        RequestFormatter.meta_get(key: encoded_key, return_cas: true, base64: base64, quiet: true,
+                                  skip_flags: raw_mode?)
       end
 
       def gat(key, ttl, options = nil)
         ttl = TtlSanitizer.sanitize(ttl)
         encoded_key, base64 = KeyRegularizer.encode(key)
-        req = RequestFormatter.meta_get(key: encoded_key, ttl: ttl, base64: base64)
+        skip_flags = raw_mode? || (options && options[:raw])
+        req = RequestFormatter.meta_get(key: encoded_key, ttl: ttl, base64: base64, skip_flags: skip_flags)
         write(req)
+        @connection_manager.flush
         response_processor.meta_get_with_value(cache_nils: cache_nils?(options))
       end
 
@@ -48,6 +55,7 @@ module Dalli
         encoded_key, base64 = KeyRegularizer.encode(key)
         req = RequestFormatter.meta_get(key: encoded_key, ttl: ttl, value: false, base64: base64)
         write(req)
+        @connection_manager.flush
         response_processor.meta_get_without_value
       end
 
@@ -57,6 +65,7 @@ module Dalli
         encoded_key, base64 = KeyRegularizer.encode(key)
         req = RequestFormatter.meta_get(key: encoded_key, value: true, return_cas: true, base64: base64)
         write(req)
+        @connection_manager.flush
         response_processor.meta_get_with_value_and_cas
       end
 
@@ -93,6 +102,7 @@ module Dalli
           return_last_access: options[:return_last_access], skip_lru_bump: options[:skip_lru_bump]
         )
         write(req)
+        @connection_manager.flush
         response_processor.meta_get_with_metadata(
           cache_nils: cache_nils?(options), return_hit_status: options[:return_hit_status],
           return_last_access: options[:return_last_access]
@@ -110,6 +120,7 @@ module Dalli
         encoded_key, base64 = KeyRegularizer.encode(key)
         req = RequestFormatter.meta_delete(key: encoded_key, cas: cas, base64: base64, stale: true)
         write(req)
+        @connection_manager.flush
         response_processor.meta_delete
       end
 
@@ -146,6 +157,7 @@ module Dalli
         write(req)
         write(value)
         write(TERMINATOR)
+        @connection_manager.flush unless quiet
       end
       # rubocop:enable Metrics/ParameterLists
 
@@ -168,6 +180,7 @@ module Dalli
         write(req)
         write(value)
         write(TERMINATOR)
+        @connection_manager.flush unless quiet?
       end
       # rubocop:enable Metrics/ParameterLists
 
@@ -177,6 +190,7 @@ module Dalli
         req = RequestFormatter.meta_delete(key: encoded_key, cas: cas,
                                            base64: base64, quiet: quiet?)
         write(req)
+        @connection_manager.flush unless quiet?
         response_processor.meta_delete unless quiet?
       end
 
@@ -202,12 +216,14 @@ module Dalli
         encoded_key, base64 = KeyRegularizer.encode(key)
         write(RequestFormatter.meta_arithmetic(key: encoded_key, delta: delta, initial: initial, incr: incr, ttl: ttl,
                                                quiet: quiet?, base64: base64))
+        @connection_manager.flush unless quiet?
         response_processor.decr_incr unless quiet?
       end
 
       # Other Commands
       def flush(delay = 0)
         write(RequestFormatter.flush(delay: delay))
+        @connection_manager.flush unless quiet?
         response_processor.flush unless quiet?
       end
 
@@ -220,21 +236,25 @@ module Dalli
 
       def stats(info = nil)
         write(RequestFormatter.stats(info))
+        @connection_manager.flush
         response_processor.stats
       end
 
       def reset_stats
         write(RequestFormatter.stats('reset'))
+        @connection_manager.flush
         response_processor.reset
       end
 
       def version
         write(RequestFormatter.version)
+        @connection_manager.flush
         response_processor.version
       end
 
       def write_noop
         write(RequestFormatter.meta_noop)
+        @connection_manager.flush
       end
 
       def authenticate_connection

data/lib/dalli/protocol/response_buffer.rb

@@ -41,6 +41,15 @@ module Dalli
         @buffer = ''.b
       end
 
+      # Ensures the buffer is initialized for reading without discarding
+      # existing data. Used by interleaved pipelined get which may have
+      # already buffered partial responses during the send phase.
+      def ensure_ready
+        return if in_progress?
+
+        @buffer = ''.b
+      end
+
       # Clear the internal response buffer
       def clear
         @buffer = nil

data/lib/dalli/protocol/value_serializer.rb

@@ -32,22 +32,8 @@ module Dalli
       end
 
       def store(value, req_options, bitflags)
-        if req_options
-          return [value.to_s, bitflags] if req_options[:raw]
-
-          # If the value is a simple string, going through serialization is costly
-          # for no benefit other than preserving encoding.
-          # Assuming most strings are either UTF-8 or BINARY we can just store
-          # that information in the bitflags.
-          if req_options[:string_fastpath] && value.instance_of?(String)
-            case value.encoding
-            when Encoding::BINARY
-              return [value, bitflags]
-            when Encoding::UTF_8
-              return [value, bitflags | FLAG_UTF8]
-            end
-          end
-        end
+        return store_raw(value, bitflags) if req_options&.dig(:raw)
+        return store_string_fastpath(value, bitflags) if use_string_fastpath?(value, req_options)
 
         [serialize_value(value), bitflags | FLAG_SERIALIZED]
       end
@@ -85,6 +71,30 @@ module Dalli
 
       private
 
+      def store_raw(value, bitflags)
+        unless value.is_a?(String)
+          raise Dalli::MarshalError, "Dalli raw mode requires string values, got: #{value.class}"
+        end
+
+        [value, bitflags]
+      end
+
+      # If the value is a simple string, going through serialization is costly
+      # for no benefit other than preserving encoding.
+      # Assuming most strings are either UTF-8 or BINARY we can just store
+      # that information in the bitflags.
+      def store_string_fastpath(value, bitflags)
+        case value.encoding
+        when Encoding::BINARY then [value, bitflags]
+        when Encoding::UTF_8 then [value, bitflags | FLAG_UTF8]
+        else [serialize_value(value), bitflags | FLAG_SERIALIZED]
+        end
+      end
+
+      def use_string_fastpath?(value, req_options)
+        req_options&.dig(:string_fastpath) && value.instance_of?(String)
+      end
+
       def warn_if_marshal_default(protocol_options)
         return if protocol_options.key?(:serializer)
         return if @@marshal_warning_logged

data/lib/dalli/socket.rb

@@ -107,7 +107,7 @@ module Dalli
         # aliases TCPSocket#initialize method to #original_resolv_initialize.
         # https://github.com/ruby/resolv-replace/blob/v0.1.1/lib/resolv-replace.rb#L21
         if RUBY_VERSION >= '3.0' &&
-           !::TCPSocket.private_instance_methods.include?(:original_resolv_initialize)
+           !::TCPSocket.private_method_defined?(:original_resolv_initialize)
           sock = new(host, port, connect_timeout: options[:socket_timeout])
           yield(sock)
         else
@@ -138,16 +138,40 @@ module Dalli
         return unless options[:socket_timeout]
 
         if sock.respond_to?(:timeout=)
+          # Ruby 3.2+ has IO#timeout for reliable cross-platform timeout handling
           sock.timeout = options[:socket_timeout]
         else
+          # Ruby 3.1 fallback using socket options
+          # struct timeval has architecture-dependent sizes (time_t, suseconds_t)
           seconds, fractional = options[:socket_timeout].divmod(1)
-          timeval = [seconds, fractional * 1_000_000].pack('l_2')
+          microseconds = (fractional * 1_000_000).to_i
+          timeval = pack_timeval(sock, seconds, microseconds)
 
           sock.setsockopt(::Socket::SOL_SOCKET, ::Socket::SO_RCVTIMEO, timeval)
           sock.setsockopt(::Socket::SOL_SOCKET, ::Socket::SO_SNDTIMEO, timeval)
         end
       end
 
+      # Pack formats for struct timeval across architectures.
+      # Uses fixed-size formats for JRuby compatibility (JRuby doesn't support _ modifier on q).
+      # - ll: 8 bytes (32-bit time_t, 32-bit suseconds_t)
+      # - qq: 16 bytes (64-bit time_t, 64-bit suseconds_t or padded 32-bit)
+      TIMEVAL_PACK_FORMATS = %w[ll qq].freeze
+      TIMEVAL_TEST_VALUES = [0, 0].freeze
+
+      # Detect and cache the correct pack format for struct timeval on this platform.
+      # Different architectures have different sizes for time_t and suseconds_t.
+      def self.timeval_pack_format(sock)
+        @timeval_pack_format ||= begin
+          expected_size = sock.getsockopt(::Socket::SOL_SOCKET, ::Socket::SO_RCVTIMEO).data.bytesize
+          TIMEVAL_PACK_FORMATS.find { |fmt| TIMEVAL_TEST_VALUES.pack(fmt).bytesize == expected_size } || 'll'
+        end
+      end
+
+      def self.pack_timeval(sock, seconds, microseconds)
+        [seconds, microseconds].pack(timeval_pack_format(sock))
+      end
+
       def self.wrapping_ssl_socket(tcp_socket, host, ssl_context)
         ssl_socket = Dalli::Socket::SSLSocket.new(tcp_socket, ssl_context)
         ssl_socket.hostname = host

data/lib/dalli/version.rb

@@ -1,7 +1,7 @@
 # frozen_string_literal: true
 
 module Dalli
-  VERSION = '4.1.0'
+  VERSION = '4.3.0'
 
   MIN_SUPPORTED_MEMCACHED_VERSION = '1.4'
 end

data/lib/dalli.rb

@@ -56,6 +56,7 @@ module Dalli
 end
 
 require_relative 'dalli/version'
+require_relative 'dalli/instrumentation'
 
 require_relative 'dalli/compressor'
 require_relative 'dalli/protocol_deprecations'

metadata

@@ -1,7 +1,7 @@
 --- !ruby/object:Gem::Specification
 name: dalli
 version: !ruby/object:Gem::Version
-  version: 4.1.0
+  version: 4.3.0
 platform: ruby
 authors:
 - Peter M. Goldstein
@@ -40,6 +40,7 @@ files:
 - lib/dalli/cas/client.rb
 - lib/dalli/client.rb
 - lib/dalli/compressor.rb
+- lib/dalli/instrumentation.rb
 - lib/dalli/key_manager.rb
 - lib/dalli/options.rb
 - lib/dalli/pid_cache.rb