dalli 4.3.3 → 5.0.1

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 57b78e30ee409a2d742fc47ee57ccdd482fe9c75f369366ae315b7ef8b649934
-  data.tar.gz: b8cad66f3cba53bbcb84b18f406eed60f22209c10dd4a312063a1e13189185ca
+  metadata.gz: 02d0fa949b7a065f86fb3ac7b511a3feacc50b700e82dcb385e0e79c56583a9d
+  data.tar.gz: d72b9e4b014ae1ae0b0d5a1ebe09ea48cdf9fbf0a2cda000efc19f17d5d34368
 SHA512:
-  metadata.gz: 18b26e3c4aa30e5f8b4195d95307afca6c9240dd1154a36966d40d52047e638758850ec06e2a40fd1bd8e69fe150dee7409b1f4a565f0a80e397aaf115315463
-  data.tar.gz: b6db69ecbc1e6d34587d8ec29b861f6a71c863b36229d3c642e3d0e646ba6234e3bac04225d64ff174d12f35b728663e82e1d2b5f7e93dc9ac3e1ff33fd58db3
+  metadata.gz: bf26484aa345df2d43a78da570027b68a7fd0dd70d7a62275ebe95a5e29ed36c11a8b1385ff0c71818f55184245e085cd75962510f9f8559aad10b0d284f8d71
+  data.tar.gz: 0b3913a33f61873d6e3da0d62ec643dbf49f679740d3469b7cb826083f02f3b32d37e2a20548634d09cecdc4389da7c5bdc43af530bb48956cb4084f91f10d9e

data/CHANGELOG.md

@@ -1,6 +1,72 @@
 Dalli Changelog
 =====================
 
+5.0.1
+==========
+
+Performance:
+
+- Reduce object allocations in pipelined get response processing (#1072, #1078)
+  - Offset-based `ResponseBuffer`: track a read offset instead of slicing a new string after every parsed response; compact only when the consumed portion exceeds 4KB and more than half the buffer
+  - Inline response processor parsing: avoid intermediate array allocations from `split`-based header parsing
+  - Block-based `pipeline_next_responses`: yield `(key, value, cas)` directly when a block is given, avoiding per-call Hash allocation
+  - `PipelinedGetter`: replace Hash-based socket-to-server mapping with linear scan (faster for typical 1-5 server counts); use `Process.clock_gettime(CLOCK_MONOTONIC)` instead of `Time.now`
+- Add cross-version benchmark script (`bin/compare_versions`) for reproducible performance comparisons across Dalli versions
+
+Bug Fixes:
+
+- Rescue `IOError` in connection manager `write`/`flush` methods (#1075)
+  - Prevents unhandled exceptions when a connection is closed mid-operation
+  - Thanks to Graham Cooper (Shopify) for this fix
+
+Development:
+
+- Add `rubocop-thread_safety` for detecting thread-safety issues (#1076)
+- Add CONTRIBUTING.md with AI contribution policy (#1074)
+
+5.0.0
+==========
+
+**Breaking Changes:**
+
+- **Removed binary protocol** - The meta protocol is now the only supported protocol
+  - The `:protocol` option is no longer used
+  - Requires memcached 1.6+ (for meta protocol support)
+  - Users on older memcached versions must upgrade or stay on Dalli 4.x
+
+- **Removed SASL authentication** - The meta protocol does not support authentication
+  - Use network-level security (firewall rules, VPN) or memcached's TLS support instead
+  - Users requiring SASL authentication must stay on Dalli 4.x with binary protocol
+
+- **Ruby 3.3+ required** - Dropped support for Ruby 3.1 and 3.2
+  - Ruby 3.2 reached end-of-life in March 2026
+  - JRuby remains supported
+
+Performance:
+
+- **~7% read performance improvement** (CRuby only)
+  - Use native `IO#read` instead of custom `readfull` implementation
+  - Enabled by Ruby 3.3's `IO#timeout=` support
+  - JRuby continues to use `readfull` for compatibility
+
+OpenTelemetry:
+
+- Migrate to stable OTel semantic conventions (#1070)
+  - `db.system` renamed to `db.system.name`
+  - `db.operation` renamed to `db.operation.name`
+  - `server.address` now contains hostname only; `server.port` is a separate integer attribute
+  - `get_with_metadata` and `fetch_with_lock` now include `server.address`/`server.port`
+- Add `db.query.text` span attribute with configurable modes
+  - `:otel_db_statement` option: `:include`, `:obfuscate`, or `nil` (default: omitted)
+- Add `peer.service` span attribute
+  - `:otel_peer_service` option for logical service naming
+
+Internal:
+
+- Simplified protocol directory structure: moved `lib/dalli/protocol/meta/*` to `lib/dalli/protocol/`
+- Removed deprecated binary protocol files and SASL authentication code
+- Removed `require 'set'` (autoloaded in Ruby 3.3+)
+
 4.3.3
 ==========
 

data/Gemfile

@@ -22,6 +22,7 @@ group :development, :test do
   gem 'rubocop-minitest'
   gem 'rubocop-performance'
   gem 'rubocop-rake'
+  gem 'rubocop-thread_safety'
   gem 'simplecov'
 end
 

data/README.md

@@ -10,26 +10,14 @@ Dalli supports:
 * Fine-grained control of data serialization and compression
 * Thread-safe operation (either through use of a connection pool, or by using the Dalli client in threadsafe mode)
 * SSL/TLS connections to memcached
-* SASL authentication
 * OpenTelemetry distributed tracing (automatic when SDK is present)
 
 The name is a variant of Salvador Dali for his famous painting [The Persistence of Memory](http://en.wikipedia.org/wiki/The_Persistence_of_Memory).
 
 ## Requirements
 
-* Ruby 3.1 or later
-* memcached 1.4 or later (1.6+ recommended for meta protocol support)
-
-## Protocol Options
-
-Dalli supports two protocols for communicating with memcached:
-
-* `:binary` (default) - Works with all memcached versions, supports SASL authentication
-* `:meta` - Requires memcached 1.6+, better performance for some operations, no authentication support
-
-```ruby
-Dalli::Client.new('localhost:11211', protocol: :meta)
-```
+* Ruby 3.3 or later (JRuby also supported)
+* memcached 1.6 or later
 
 ## Configuration Options
 
@@ -64,7 +52,7 @@ By default, Dalli uses Ruby's Marshal for serialization. Deserializing untrusted
 Dalli::Client.new('localhost:11211', serializer: JSON)
 ```
 
-See the [4.0-Upgrade.md](4.0-Upgrade.md) guide for more information.
+See the [5.0-Upgrade.md](5.0-Upgrade.md) guide for upgrade information.
 
 ## OpenTelemetry Tracing
 
@@ -125,7 +113,7 @@ To install this gem onto your local machine, run `bundle exec rake install`.
 
 ## Contributing
 
-If you have a fix you wish to provide, please fork the code, fix in your local project and then send a pull request on github.  Please ensure that you include a test which verifies your fix and update the [changelog](CHANGELOG.md) with a one sentence description of your fix so you get credit as a contributor.
+Please see [CONTRIBUTING.md](CONTRIBUTING.md) for guidelines on how to contribute, including our policy on AI-authored contributions.
 
 ## Appreciation
 

data/lib/dalli/client.rb

@@ -1,7 +1,6 @@
 # frozen_string_literal: true
 
 require 'digest/md5'
-require 'set'
 
 # encoding: ascii
 module Dalli
@@ -48,8 +47,6 @@ module Dalli
     #                 serialization pipeline.
     # - :digest_class - defaults to Digest::MD5, allows you to pass in an object that responds to the hexdigest method,
     #                   useful for injecting a FIPS compliant hash object.
-    # - :protocol - one of either :binary or :meta, defaulting to :binary.  This sets the protocol that Dalli uses
-    #               to communicate with memcached.
     # - :otel_db_statement - controls the +db.query.text+ span attribute when OpenTelemetry is loaded.
     #                        +:include+ logs the full operation and key(s), +:obfuscate+ replaces keys with "?",
     #                        +nil+ (default) omits the attribute entirely.
@@ -58,9 +55,9 @@ module Dalli
     def initialize(servers = nil, options = {})
       @normalized_servers = ::Dalli::ServersArgNormalizer.normalize_servers(servers)
       @options = normalize_options(options)
+      warn_removed_options(@options)
       @key_manager = ::Dalli::KeyManager.new(@options)
       @ring = nil
-      emit_deprecation_warnings
     end
 
     #
@@ -102,10 +99,7 @@ module Dalli
     end
 
     ##
-    # Get value with extended metadata using the meta protocol.
-    #
-    # IMPORTANT: This method requires memcached 1.6+ and the meta protocol (protocol: :meta).
-    # It will raise an error if used with the binary protocol.
+    # Get value with extended metadata.
     #
     # @param key [String] the cache key
     # @param options [Hash] options controlling what metadata to return
@@ -133,8 +127,6 @@ module Dalli
     #   # => { value: "data", cas: 123, hit_before: true, last_access: 42 }
     #
     def get_with_metadata(key, options = {})
-      raise_unless_meta_protocol!
-
       key = key.to_s
       key = @key_manager.validate_key(key)
 
@@ -208,9 +200,6 @@ module Dalli
     # cache entry (the "thundering herd" problem). Only one client wins the right to
     # regenerate; other clients receive the stale value (if available) or wait.
     #
-    # IMPORTANT: This method requires memcached 1.6+ and the meta protocol (protocol: :meta).
-    # It will raise an error if used with the binary protocol.
-    #
     # @param key [String] the cache key
     # @param ttl [Integer] time-to-live for the cached value in seconds
     # @param lock_ttl [Integer] how long the lock/stub lives (default: 30 seconds)
@@ -236,8 +225,6 @@ module Dalli
     def fetch_with_lock(key, ttl: nil, lock_ttl: 30, recache_threshold: nil, req_options: nil, &block)
       raise ArgumentError, 'Block is required for fetch_with_lock' unless block_given?
 
-      raise_unless_meta_protocol!
-
       key = key.to_s
       key = @key_manager.validate_key(key)
 
@@ -514,10 +501,6 @@ module Dalli
 
     private
 
-    # Records hit/miss metrics on a span for cache observability.
-    # @param span [OpenTelemetry::Trace::Span, nil] the span to record on
-    # @param key_count [Integer] total keys requested
-    # @param hit_count [Integer] keys found in cache
     def record_hit_miss_metrics(span, key_count, hit_count)
       return unless span
 
@@ -607,16 +590,7 @@ module Dalli
     end
 
     def ring
-      @ring ||= Dalli::Ring.new(@normalized_servers, protocol_implementation, @options)
-    end
-
-    def protocol_implementation
-      @protocol_implementation ||= case @options[:protocol]&.to_s
-                                   when 'meta'
-                                     Dalli::Protocol::Meta
-                                   else
-                                     Dalli::Protocol::Binary
-                                   end
+      @ring ||= Dalli::Ring.new(@normalized_servers, @options)
     end
 
     ##
@@ -627,7 +601,7 @@ module Dalli
     #
     # This method also forces retries on network errors - when
     # a particular memcached instance becomes unreachable, or the
-    # operational times out.
+    # operation times out.
     ##
     def perform(*all_args)
       return yield if block_given?
@@ -654,6 +628,21 @@ module Dalli
       raise ArgumentError, "cannot convert :expires_in => #{opts[:expires_in].inspect} to an integer"
     end
 
+    REMOVED_OPTIONS = {
+      protocol: 'Dalli 5.0 only supports the meta protocol. The :protocol option has been removed.',
+      username: 'Dalli 5.0 removed SASL authentication support. The :username option is ignored.',
+      password: 'Dalli 5.0 removed SASL authentication support. The :password option is ignored.'
+    }.freeze
+    private_constant :REMOVED_OPTIONS
+
+    def warn_removed_options(opts)
+      REMOVED_OPTIONS.each do |key, message|
+        next unless opts.key?(key)
+
+        Dalli.logger.warn(message)
+      end
+    end
+
     def pipelined_getter
       PipelinedGetter.new(ring, @key_manager)
     end
@@ -665,15 +654,5 @@ module Dalli
     def pipelined_deleter
       PipelinedDeleter.new(ring, @key_manager)
     end
-
-    def raise_unless_meta_protocol!
-      return if protocol_implementation == Dalli::Protocol::Meta
-
-      raise Dalli::DalliError,
-            'This operation requires the meta protocol (memcached 1.6+). ' \
-            'Use protocol: :meta when creating the client.'
-    end
-
-    include ProtocolDeprecations
   end
 end

data/lib/dalli/instrumentation.rb

@@ -8,7 +8,7 @@ module Dalli
   # When OpenTelemetry is loaded, Dalli automatically creates spans for cache operations.
   # When OpenTelemetry is not available, all tracing methods are no-ops with zero overhead.
   #
-  # Dalli 4.3.2 uses the stable OTel semantic conventions for database spans.
+  # Dalli 5.0 uses the stable OTel semantic conventions for database spans.
   #
   # == Span Attributes
   #
@@ -61,11 +61,13 @@ module Dalli
       # Uses the library name 'dalli' and current Dalli::VERSION.
       #
       # @return [OpenTelemetry::Trace::Tracer, nil] the tracer or nil if OTel unavailable
+      # rubocop:disable ThreadSafety/ClassInstanceVariable
       def tracer
         return @tracer if defined?(@tracer)
 
         @tracer = (OpenTelemetry.tracer_provider.tracer('dalli', Dalli::VERSION) if defined?(OpenTelemetry))
       end
+      # rubocop:enable ThreadSafety/ClassInstanceVariable
 
       # Returns true if instrumentation is enabled (OpenTelemetry SDK is available).
       #

data/lib/dalli/options.rb

@@ -6,7 +6,7 @@ module Dalli
   # Make Dalli threadsafe by using a lock around all
   # public server methods.
   #
-  # Dalli::Protocol::Binary.extend(Dalli::Threadsafe)
+  # Dalli::Protocol::Meta.extend(Dalli::Threadsafe)
   #
   module Threadsafe
     def self.extended(obj)

data/lib/dalli/pid_cache.rb

@@ -13,7 +13,7 @@ module Dalli
         attr_reader :pid
 
         def update!
-          @pid = Process.pid
+          @pid = Process.pid # rubocop:disable ThreadSafety/ClassInstanceVariable
         end
       end
       update!

data/lib/dalli/pipelined_getter.rb

@@ -1,7 +1,5 @@
 # frozen_string_literal: true
 
-require 'set'
-
 module Dalli
   ##
   # Contains logic for the pipelined gets implemented by the client.

data/lib/dalli/protocol/base.rb

@@ -22,6 +22,7 @@ module Dalli
 
       def initialize(attribs, client_options = {})
         hostname, port, socket_type, @weight, user_creds = ServerConfigParser.parse(attribs)
+        warn_uri_credentials(user_creds)
         @options = client_options.merge(user_creds)
         @raw_mode = client_options[:raw]
         @value_marshaller = @raw_mode ? StringMarshaller.new(@options) : ValueMarshaller.new(@options)
@@ -152,18 +153,6 @@ module Dalli
         !response_buffer.in_progress?
       end
 
-      def username
-        @options[:username] || ENV.fetch('MEMCACHE_USERNAME', nil)
-      end
-
-      def password
-        @options[:password] || ENV.fetch('MEMCACHE_PASSWORD', nil)
-      end
-
-      def require_auth?
-        !username.nil?
-      end
-
       def quiet?
         Thread.current[::Dalli::QUIET]
       end
@@ -173,6 +162,16 @@ module Dalli
 
       private
 
+      URI_CREDENTIAL_WARNING = 'Dalli 5.0 removed SASL authentication. ' \
+                               'Credentials in memcached:// URIs are ignored.'
+      private_constant :URI_CREDENTIAL_WARNING
+
+      def warn_uri_credentials(user_creds)
+        return if user_creds[:username].nil? && user_creds[:password].nil?
+
+        Dalli.logger.warn(URI_CREDENTIAL_WARNING)
+      end
+
       ALLOWED_QUIET_OPS = %i[add replace set delete incr decr append prepend flush noop].freeze
       private_constant :ALLOWED_QUIET_OPS
 
@@ -227,8 +226,7 @@ module Dalli
 
       def connect
         @connection_manager.establish_connection
-        authenticate_connection if require_auth?
-        @version = version # Connect socket if not authed
+        @version = version
         up!
       end
 

data/lib/dalli/protocol/connection_manager.rb

@@ -156,20 +156,26 @@ module Dalli
       end
 
       def read(count)
-        @sock.readfull(count)
+        # JRuby doesn't support IO#timeout=, so use custom readfull implementation
+        # CRuby 3.3+ has IO#timeout= which makes IO#read work with timeouts
+        if RUBY_ENGINE == 'jruby'
+          @sock.readfull(count)
+        else
+          @sock.read(count)
+        end
       rescue SystemCallError, *TIMEOUT_ERRORS, *SSL_ERRORS, EOFError => e
         error_on_request!(e)
       end
 
       def write(bytes)
         @sock.write(bytes)
-      rescue SystemCallError, *TIMEOUT_ERRORS, *SSL_ERRORS => e
+      rescue SystemCallError, *TIMEOUT_ERRORS, *SSL_ERRORS, IOError => e
         error_on_request!(e)
       end
 
       def flush
         @sock.flush
-      rescue SystemCallError, *TIMEOUT_ERRORS, *SSL_ERRORS => e
+      rescue SystemCallError, *TIMEOUT_ERRORS, *SSL_ERRORS, IOError => e
         error_on_request!(e)
       end
 

data/lib/dalli/protocol/meta.rb

@@ -257,13 +257,9 @@ module Dalli
         @connection_manager.flush
       end
 
-      def authenticate_connection
-        raise Dalli::DalliError, 'Authentication not supported for the meta protocol.'
-      end
-
-      require_relative 'meta/key_regularizer'
-      require_relative 'meta/request_formatter'
-      require_relative 'meta/response_processor'
+      require_relative 'key_regularizer'
+      require_relative 'request_formatter'
+      require_relative 'response_processor'
     end
   end
 end

data/lib/dalli/protocol/server_config_parser.rb

@@ -6,7 +6,7 @@ module Dalli
   module Protocol
     ##
     # Dalli::Protocol::ServerConfigParser parses a server string passed to
-    # a Dalli::Protocol::Binary instance into the hostname, port, weight, and
+    # a Dalli::Protocol::Meta instance into the hostname, port, weight, and
     # socket_type.
     ##
     class ServerConfigParser

data/lib/dalli/ring.rb

@@ -23,9 +23,9 @@ module Dalli
 
     attr_accessor :servers, :continuum
 
-    def initialize(servers_arg, protocol_implementation, options)
+    def initialize(servers_arg, options)
       @servers = servers_arg.map do |s|
-        protocol_implementation.new(s, options)
+        Dalli::Protocol::Meta.new(s, options)
       end
       @continuum = nil
       @continuum = build_continuum(servers) if servers.size > 1

data/lib/dalli/servers_arg_normalizer.rb

@@ -16,7 +16,7 @@ module Dalli
   #     weight are optional (e.g. 'localhost', 'abc.com:12345', 'example.org:22222:3')
   #   * A colon separated string of (UNIX socket, weight) where the weight is optional
   #     (e.g. '/var/run/memcached/socket', '/tmp/xyz:3') (not supported on Windows)
-  #   * A URI with a 'memcached' protocol, which will typically include a username/password
+  #   * A URI with a 'memcached' protocol (e.g. 'memcached://localhost:11211')
   #
   # The methods in this module do not validate the format of individual server strings, but
   # rather normalize the argument into a compact array, wherein each array entry corresponds

data/lib/dalli/socket.rb

@@ -108,12 +108,14 @@ module Dalli
       # Detect and cache whether TCPSocket supports the connect_timeout: keyword argument.
       # Returns false if TCPSocket#initialize has been monkey-patched by gems like
       # socksify or resolv-replace, which don't support keyword arguments.
+      # rubocop:disable ThreadSafety/ClassInstanceVariable
       def self.supports_connect_timeout?
         return @supports_connect_timeout if defined?(@supports_connect_timeout)
 
         @supports_connect_timeout = RUBY_VERSION >= '3.0' &&
                                     ::TCPSocket.instance_method(:initialize).parameters == TCPSOCKET_NATIVE_PARAMETERS
       end
+      # rubocop:enable ThreadSafety/ClassInstanceVariable
 
       def self.create_socket_with_timeout(host, port, options)
         if supports_connect_timeout?
@@ -170,12 +172,14 @@ module Dalli
 
       # Detect and cache the correct pack format for struct timeval on this platform.
       # Different architectures have different sizes for time_t and suseconds_t.
+      # rubocop:disable ThreadSafety/ClassInstanceVariable
       def self.timeval_pack_format(sock)
         @timeval_pack_format ||= begin
           expected_size = sock.getsockopt(::Socket::SOL_SOCKET, ::Socket::SO_RCVTIMEO).data.bytesize
           TIMEVAL_PACK_FORMATS.find { |fmt| TIMEVAL_TEST_VALUES.pack(fmt).bytesize == expected_size } || 'll'
         end
       end
+      # rubocop:enable ThreadSafety/ClassInstanceVariable
 
       def self.pack_timeval(sock, seconds, microseconds)
         [seconds, microseconds].pack(timeval_pack_format(sock))

data/lib/dalli/version.rb

@@ -1,7 +1,7 @@
 # frozen_string_literal: true
 
 module Dalli
-  VERSION = '4.3.3'
+  VERSION = '5.0.1'
 
-  MIN_SUPPORTED_MEMCACHED_VERSION = '1.4'
+  MIN_SUPPORTED_MEMCACHED_VERSION = '1.6'
 end

data/lib/dalli.rb

@@ -38,7 +38,7 @@ module Dalli
   QUIET = :dalli_multi
 
   def self.logger
-    @logger ||= rails_logger || default_logger
+    @logger ||= rails_logger || default_logger # rubocop:disable ThreadSafety/ClassInstanceVariable
   end
 
   def self.rails_logger
@@ -54,7 +54,7 @@ module Dalli
   end
 
   def self.logger=(logger)
-    @logger = logger
+    @logger = logger # rubocop:disable ThreadSafety/ClassInstanceVariable
   end
 end
 
@@ -62,7 +62,6 @@ require_relative 'dalli/version'
 require_relative 'dalli/instrumentation'
 
 require_relative 'dalli/compressor'
-require_relative 'dalli/protocol_deprecations'
 require_relative 'dalli/client'
 require_relative 'dalli/key_manager'
 require_relative 'dalli/pipelined_getter'
@@ -71,7 +70,6 @@ require_relative 'dalli/pipelined_deleter'
 require_relative 'dalli/ring'
 require_relative 'dalli/protocol'
 require_relative 'dalli/protocol/base'
-require_relative 'dalli/protocol/binary'
 require_relative 'dalli/protocol/connection_manager'
 require_relative 'dalli/protocol/meta'
 require_relative 'dalli/protocol/response_buffer'

metadata

@@ -1,7 +1,7 @@
 --- !ruby/object:Gem::Specification
 name: dalli
 version: !ruby/object:Gem::Version
-  version: 4.3.3
+  version: 5.0.1
 platform: ruby
 authors:
 - Peter M. Goldstein
@@ -49,24 +49,18 @@ files:
 - lib/dalli/pipelined_setter.rb
 - lib/dalli/protocol.rb
 - lib/dalli/protocol/base.rb
-- lib/dalli/protocol/binary.rb
-- lib/dalli/protocol/binary/request_formatter.rb
-- lib/dalli/protocol/binary/response_header.rb
-- lib/dalli/protocol/binary/response_processor.rb
-- lib/dalli/protocol/binary/sasl_authentication.rb
 - lib/dalli/protocol/connection_manager.rb
+- lib/dalli/protocol/key_regularizer.rb
 - lib/dalli/protocol/meta.rb
-- lib/dalli/protocol/meta/key_regularizer.rb
-- lib/dalli/protocol/meta/request_formatter.rb
-- lib/dalli/protocol/meta/response_processor.rb
+- lib/dalli/protocol/request_formatter.rb
 - lib/dalli/protocol/response_buffer.rb
+- lib/dalli/protocol/response_processor.rb
 - lib/dalli/protocol/server_config_parser.rb
 - lib/dalli/protocol/string_marshaller.rb
 - lib/dalli/protocol/ttl_sanitizer.rb
 - lib/dalli/protocol/value_compressor.rb
 - lib/dalli/protocol/value_marshaller.rb
 - lib/dalli/protocol/value_serializer.rb
-- lib/dalli/protocol_deprecations.rb
 - lib/dalli/ring.rb
 - lib/dalli/servers_arg_normalizer.rb
 - lib/dalli/socket.rb
@@ -77,7 +71,7 @@ licenses:
 - MIT
 metadata:
   bug_tracker_uri: https://github.com/petergoldstein/dalli/issues
-  changelog_uri: https://github.com/petergoldstein/dalli/blob/v4.3/CHANGELOG.md
+  changelog_uri: https://github.com/petergoldstein/dalli/blob/main/CHANGELOG.md
   rubygems_mfa_required: 'true'
 rdoc_options: []
 require_paths:
@@ -86,7 +80,7 @@ required_ruby_version: !ruby/object:Gem::Requirement
   requirements:
   - - ">="
     - !ruby/object:Gem::Version
-      version: '3.1'
+      version: '3.3'
 required_rubygems_version: !ruby/object:Gem::Requirement
   requirements:
   - - ">="

data/lib/dalli/protocol/binary/request_formatter.rb

@@ -1,117 +0,0 @@
-# frozen_string_literal: true
-
-module Dalli
-  module Protocol
-    class Binary
-      ##
-      # Class that encapsulates logic for formatting binary protocol requests
-      # to memcached.
-      ##
-      class RequestFormatter
-        REQUEST = 0x80
-
-        OPCODES = {
-          get: 0x00,
-          set: 0x01,
-          add: 0x02,
-          replace: 0x03,
-          delete: 0x04,
-          incr: 0x05,
-          decr: 0x06,
-          flush: 0x08,
-          noop: 0x0A,
-          version: 0x0B,
-          getkq: 0x0D,
-          append: 0x0E,
-          prepend: 0x0F,
-          stat: 0x10,
-          setq: 0x11,
-          addq: 0x12,
-          replaceq: 0x13,
-          deleteq: 0x14,
-          incrq: 0x15,
-          decrq: 0x16,
-          flushq: 0x18,
-          appendq: 0x19,
-          prependq: 0x1A,
-          touch: 0x1C,
-          gat: 0x1D,
-          auth_negotiation: 0x20,
-          auth_request: 0x21,
-          auth_continue: 0x22
-        }.freeze
-
-        REQ_HEADER_FORMAT = 'CCnCCnNNQ'
-
-        KEY_ONLY = 'a*'
-        TTL_AND_KEY = 'Na*'
-        KEY_AND_VALUE = 'a*a*'
-        INCR_DECR = 'NNNNNa*'
-        TTL_ONLY = 'N'
-        NO_BODY = ''
-
-        BODY_FORMATS = {
-          get: KEY_ONLY,
-          getkq: KEY_ONLY,
-          delete: KEY_ONLY,
-          deleteq: KEY_ONLY,
-          stat: KEY_ONLY,
-
-          append: KEY_AND_VALUE,
-          prepend: KEY_AND_VALUE,
-          appendq: KEY_AND_VALUE,
-          prependq: KEY_AND_VALUE,
-          auth_request: KEY_AND_VALUE,
-          auth_continue: KEY_AND_VALUE,
-
-          set: 'NNa*a*',
-          setq: 'NNa*a*',
-          add: 'NNa*a*',
-          addq: 'NNa*a*',
-          replace: 'NNa*a*',
-          replaceq: 'NNa*a*',
-
-          incr: INCR_DECR,
-          decr: INCR_DECR,
-          incrq: INCR_DECR,
-          decrq: INCR_DECR,
-
-          flush: TTL_ONLY,
-          flushq: TTL_ONLY,
-
-          noop: NO_BODY,
-          auth_negotiation: NO_BODY,
-          version: NO_BODY,
-
-          touch: TTL_AND_KEY,
-          gat: TTL_AND_KEY
-        }.freeze
-        FORMAT = BODY_FORMATS.transform_values { |v| REQ_HEADER_FORMAT + v }
-
-        # rubocop:disable Metrics/ParameterLists
-        def self.standard_request(opkey:, key: nil, value: nil, opaque: 0, cas: 0, bitflags: nil, ttl: nil)
-          extra_len = (bitflags.nil? ? 0 : 4) + (ttl.nil? ? 0 : 4)
-          key_len = key.nil? ? 0 : key.bytesize
-          value_len = value.nil? ? 0 : value.bytesize
-          header = [REQUEST, OPCODES[opkey], key_len, extra_len, 0, 0, extra_len + key_len + value_len, opaque, cas]
-          body = [bitflags, ttl, key, value].compact
-          (header + body).pack(FORMAT[opkey])
-        end
-        # rubocop:enable Metrics/ParameterLists
-
-        def self.decr_incr_request(opkey:, key: nil, count: nil, initial: nil, expiry: nil)
-          extra_len = 20
-          (h, l) = as_8byte_uint(count)
-          (dh, dl) = as_8byte_uint(initial)
-          header = [REQUEST, OPCODES[opkey], key.bytesize, extra_len, 0, 0, key.bytesize + extra_len, 0, 0]
-          body = [h, l, dh, dl, expiry, key]
-          (header + body).pack(FORMAT[opkey])
-        end
-
-        def self.as_8byte_uint(val)
-          [val >> 32, val & 0xFFFFFFFF]
-        end
-      end
-    end
-  end
-end

data/lib/dalli/protocol/binary/response_header.rb

@@ -1,36 +0,0 @@
-# frozen_string_literal: true
-
-module Dalli
-  module Protocol
-    class Binary
-      ##
-      # Class that encapsulates data parsed from a memcached response header.
-      ##
-      class ResponseHeader
-        SIZE = 24
-        FMT = '@2nCCnNNQ'
-
-        attr_reader :key_len, :extra_len, :data_type, :status, :body_len, :opaque, :cas
-
-        def initialize(buf)
-          raise ArgumentError, "Response buffer must be at least #{SIZE} bytes" unless buf.bytesize >= SIZE
-
-          @key_len, @extra_len, @data_type, @status, @body_len, @opaque, @cas = buf.unpack(FMT)
-        end
-
-        def ok?
-          status.zero?
-        end
-
-        def not_found?
-          status == 1
-        end
-
-        NOT_STORED_STATUSES = [2, 5].freeze
-        def not_stored?
-          NOT_STORED_STATUSES.include?(status)
-        end
-      end
-    end
-  end
-end

data/lib/dalli/protocol/binary/response_processor.rb

@@ -1,229 +0,0 @@
-# frozen_string_literal: true
-
-module Dalli
-  module Protocol
-    class Binary
-      ##
-      # Class that encapsulates logic for processing binary protocol responses
-      # from memcached.  Includes logic for pulling data from an IO source
-      # and parsing into local values.  Handles errors on unexpected values.
-      ##
-      class ResponseProcessor
-        # Response codes taken from:
-        # https://github.com/memcached/memcached/wiki/BinaryProtocolRevamped#response-status
-        RESPONSE_CODES = {
-          0 => 'No error',
-          1 => 'Key not found',
-          2 => 'Key exists',
-          3 => 'Value too large',
-          4 => 'Invalid arguments',
-          5 => 'Item not stored',
-          6 => 'Incr/decr on a non-numeric value',
-          7 => 'The vbucket belongs to another server',
-          8 => 'Authentication error',
-          9 => 'Authentication continue',
-          0x20 => 'Authentication required',
-          0x81 => 'Unknown command',
-          0x82 => 'Out of memory',
-          0x83 => 'Not supported',
-          0x84 => 'Internal error',
-          0x85 => 'Busy',
-          0x86 => 'Temporary failure'
-        }.freeze
-
-        def initialize(io_source, value_marshaller)
-          @io_source = io_source
-          @value_marshaller = value_marshaller
-        end
-
-        def read(num_bytes)
-          @io_source.read(num_bytes)
-        end
-
-        def read_response
-          resp_header = ResponseHeader.new(read_header)
-          body = read(resp_header.body_len) if resp_header.body_len.positive?
-          [resp_header, body]
-        end
-
-        def unpack_response_body(resp_header, body, parse_as_stored_value)
-          extra_len = resp_header.extra_len
-          key_len = resp_header.key_len
-          bitflags = extra_len.positive? ? body.unpack1('N') : 0x0
-          key = body.byteslice(extra_len, key_len).force_encoding(Encoding::UTF_8) if key_len.positive?
-          value = body.byteslice((extra_len + key_len)..-1)
-          value = @value_marshaller.retrieve(value, bitflags) if parse_as_stored_value
-          [key, value]
-        end
-
-        def read_header
-          read(ResponseHeader::SIZE) || raise(Dalli::NetworkError, 'No response')
-        end
-
-        def raise_on_not_ok!(resp_header)
-          return if resp_header.ok?
-
-          raise Dalli::DalliError, "Response error #{resp_header.status}: #{RESPONSE_CODES[resp_header.status]}"
-        end
-
-        def get(cache_nils: false)
-          resp_header, body = read_response
-
-          return false if resp_header.not_stored? # Not stored, normal status for add operation
-          return cache_nils ? ::Dalli::NOT_FOUND : nil if resp_header.not_found?
-
-          raise_on_not_ok!(resp_header)
-          return true unless body
-
-          unpack_response_body(resp_header, body, true).last
-        end
-
-        ##
-        # Response for a storage operation.  Returns the cas on success.  False
-        # if the value wasn't stored.  And raises an error on all other error
-        # codes from memcached.
-        ##
-        def storage_response
-          resp_header, = read_response
-          return nil if resp_header.not_found?
-          return false if resp_header.not_stored? # Not stored, normal status for add operation
-
-          raise_on_not_ok!(resp_header)
-          resp_header.cas
-        end
-
-        def delete
-          resp_header, = read_response
-          return false if resp_header.not_found? || resp_header.not_stored?
-
-          raise_on_not_ok!(resp_header)
-          true
-        end
-
-        def data_cas_response
-          resp_header, body = read_response
-          return [nil, resp_header.cas] if resp_header.not_found?
-          return [nil, false] if resp_header.not_stored?
-
-          raise_on_not_ok!(resp_header)
-          return [nil, resp_header.cas] unless body
-
-          [unpack_response_body(resp_header, body, true).last, resp_header.cas]
-        end
-
-        # Returns the new value for the key, if found and updated
-        def decr_incr
-          body = generic_response
-          body ? body.unpack1('Q>') : body
-        end
-
-        def stats
-          hash = {}
-          loop do
-            resp_header, body = read_response
-            # This is the response to the terminating noop / end of stat
-            return hash if resp_header.ok? && resp_header.key_len.zero?
-
-            # Ignore any responses with non-zero status codes,
-            # such as errors from set operations.  That allows
-            # this code to be used at the end of a multi
-            # block to clear any error responses from inside the multi.
-            next unless resp_header.ok?
-
-            key, value = unpack_response_body(resp_header, body, true)
-            hash[key] = value
-          end
-        end
-
-        def flush
-          no_body_response
-        end
-
-        def reset
-          generic_response
-        end
-
-        def version
-          generic_response
-        end
-
-        def consume_all_responses_until_noop
-          loop do
-            resp_header, = read_response
-            # This is the response to the terminating noop / end of stat
-            return true if resp_header.ok? && resp_header.key_len.zero?
-          end
-        end
-
-        def generic_response
-          resp_header, body = read_response
-
-          return false if resp_header.not_stored? # Not stored, normal status for add operation
-          return nil if resp_header.not_found?
-
-          raise_on_not_ok!(resp_header)
-          return true unless body
-
-          unpack_response_body(resp_header, body, false).last
-        end
-
-        def no_body_response
-          resp_header, = read_response
-          return false if resp_header.not_stored? # Not stored, possible status for append/prepend/delete
-
-          raise_on_not_ok!(resp_header)
-          true
-        end
-
-        def validate_auth_format(extra_len, count)
-          return if extra_len.zero?
-
-          raise Dalli::NetworkError, "Unexpected message format: #{extra_len} #{count}"
-        end
-
-        def auth_response(buf = read_header)
-          resp_header = ResponseHeader.new(buf)
-          body_len = resp_header.body_len
-          validate_auth_format(resp_header.extra_len, body_len)
-          content = read(body_len) if body_len.positive?
-          [resp_header.status, content]
-        end
-
-        ##
-        # This method returns an array of values used in a pipelined
-        # getk process.  The first value is the number of bytes by
-        # which to advance the pointer in the buffer.  If the
-        # complete response is found in the buffer, this will
-        # be the response size.  Otherwise it is zero.
-        #
-        # The remaining three values in the array are the ResponseHeader,
-        # key, and value.
-        ##
-        def getk_response_from_buffer(buf, offset = 0)
-          # There's no header in the buffer, so don't advance
-          return [0, nil, nil, nil, nil] unless buf && buf.bytesize >= offset + ResponseHeader::SIZE
-
-          resp_header = ResponseHeader.new(buf.byteslice(offset, ResponseHeader::SIZE))
-          body_len = resp_header.body_len
-
-          # We have a complete response that has no body.
-          # This is either the response to the terminating
-          # noop or, if the status is not zero, an intermediate
-          # error response that needs to be discarded.
-          return [ResponseHeader::SIZE, resp_header.ok?, resp_header.cas, nil, nil] if body_len.zero?
-
-          resp_size = ResponseHeader::SIZE + body_len
-          # The header is in the buffer, but the body is not.  As we don't have
-          # a complete response, don't advance the buffer
-          return [0, nil, nil, nil, nil] unless buf.bytesize >= offset + resp_size
-
-          # The full response is in our buffer, so parse it and return
-          # the values
-          body = buf.byteslice(offset + ResponseHeader::SIZE, body_len)
-          key, value = unpack_response_body(resp_header, body, true)
-          [resp_size, resp_header.ok?, resp_header.cas, key, value]
-        end
-      end
-    end
-  end
-end

data/lib/dalli/protocol/binary/sasl_authentication.rb

@@ -1,60 +0,0 @@
-# frozen_string_literal: true
-
-module Dalli
-  module Protocol
-    class Binary
-      ##
-      # Code to support SASL authentication
-      ##
-      module SaslAuthentication
-        def perform_auth_negotiation
-          write(RequestFormatter.standard_request(opkey: :auth_negotiation))
-
-          status, content = response_processor.auth_response
-          return [status, []] if content.nil?
-
-          # Substitute spaces for the \x00 returned by
-          # memcached as a separator for easier
-          content&.tr!("\u0000", ' ')
-          mechanisms = content&.split
-          [status, mechanisms]
-        end
-
-        PLAIN_AUTH = 'PLAIN'
-
-        def supported_mechanisms!(mechanisms)
-          unless mechanisms.include?(PLAIN_AUTH)
-            raise NotImplementedError,
-                  'Dalli only supports the PLAIN authentication mechanism'
-          end
-          [PLAIN_AUTH]
-        end
-
-        def authenticate_with_plain
-          write(RequestFormatter.standard_request(opkey: :auth_request,
-                                                  key: PLAIN_AUTH,
-                                                  value: "\x0#{username}\x0#{password}"))
-          @response_processor.auth_response
-        end
-
-        def authenticate_connection
-          Dalli.logger.info { "Dalli/SASL authenticating as #{username}" }
-
-          status, mechanisms = perform_auth_negotiation
-          return Dalli.logger.debug('Authentication not required/supported by server') if status == 0x81
-
-          supported_mechanisms!(mechanisms)
-          status, content = authenticate_with_plain
-
-          return Dalli.logger.info("Dalli/SASL: #{content}") if status.zero?
-
-          raise Dalli::DalliError, "Error authenticating: 0x#{status.to_s(16)}" unless status == 0x21
-
-          raise NotImplementedError, 'No two-step authentication mechanisms supported'
-          # (step, msg) = sasl.receive('challenge', content)
-          # raise Dalli::NetworkError, "Authentication failed" if sasl.failed? || step != 'response'
-        end
-      end
-    end
-  end
-end

data/lib/dalli/protocol/binary.rb

@@ -1,200 +0,0 @@
-# frozen_string_literal: true
-
-require 'forwardable'
-require 'socket'
-require 'timeout'
-
-module Dalli
-  module Protocol
-    ##
-    # Access point for a single Memcached server, accessed via Memcached's binary
-    # protocol.  Contains logic for managing connection state to the server (retries, etc),
-    # formatting requests to the server, and unpacking responses.
-    ##
-    class Binary < Base
-      def response_processor
-        @response_processor ||= ResponseProcessor.new(@connection_manager, @value_marshaller)
-      end
-
-      private
-
-      # Retrieval Commands
-      def get(key, options = nil)
-        req = RequestFormatter.standard_request(opkey: :get, key: key)
-        write(req)
-        @connection_manager.flush
-        response_processor.get(cache_nils: cache_nils?(options))
-      end
-
-      def quiet_get_request(key)
-        RequestFormatter.standard_request(opkey: :getkq, key: key)
-      end
-
-      def gat(key, ttl, options = nil)
-        ttl = TtlSanitizer.sanitize(ttl)
-        req = RequestFormatter.standard_request(opkey: :gat, key: key, ttl: ttl)
-        write(req)
-        @connection_manager.flush
-        response_processor.get(cache_nils: cache_nils?(options))
-      end
-
-      def touch(key, ttl)
-        ttl = TtlSanitizer.sanitize(ttl)
-        write(RequestFormatter.standard_request(opkey: :touch, key: key, ttl: ttl))
-        @connection_manager.flush
-        response_processor.generic_response
-      end
-
-      # TODO: This is confusing, as there's a cas command in memcached
-      # and this isn't it.  Maybe rename?  Maybe eliminate?
-      def cas(key)
-        req = RequestFormatter.standard_request(opkey: :get, key: key)
-        write(req)
-        @connection_manager.flush
-        response_processor.data_cas_response
-      end
-
-      # Storage Commands
-      def set(key, value, ttl, cas, options)
-        opkey = quiet? ? :setq : :set
-        storage_req(opkey, key, value, ttl, cas, options)
-      end
-
-      # Pipelined set - writes a quiet set request without reading response.
-      # Used by PipelinedSetter for bulk operations.
-      def pipelined_set(key, value, ttl, options)
-        storage_req(:setq, key, value, ttl, 0, options)
-      end
-
-      def add(key, value, ttl, options)
-        opkey = quiet? ? :addq : :add
-        storage_req(opkey, key, value, ttl, 0, options)
-      end
-
-      def replace(key, value, ttl, cas, options)
-        opkey = quiet? ? :replaceq : :replace
-        storage_req(opkey, key, value, ttl, cas, options)
-      end
-
-      # rubocop:disable Metrics/ParameterLists
-      def storage_req(opkey, key, value, ttl, cas, options)
-        (value, bitflags) = @value_marshaller.store(key, value, options)
-        ttl = TtlSanitizer.sanitize(ttl)
-
-        req = RequestFormatter.standard_request(opkey: opkey, key: key,
-                                                value: value, bitflags: bitflags,
-                                                ttl: ttl, cas: cas)
-        write(req)
-        @connection_manager.flush unless quiet?
-        response_processor.storage_response unless quiet?
-      end
-      # rubocop:enable Metrics/ParameterLists
-
-      def append(key, value)
-        opkey = quiet? ? :appendq : :append
-        write_append_prepend opkey, key, value
-      end
-
-      def prepend(key, value)
-        opkey = quiet? ? :prependq : :prepend
-        write_append_prepend opkey, key, value
-      end
-
-      def write_append_prepend(opkey, key, value)
-        write(RequestFormatter.standard_request(opkey: opkey, key: key, value: value))
-        @connection_manager.flush unless quiet?
-        response_processor.no_body_response unless quiet?
-      end
-
-      # Delete Commands
-      def delete(key, cas)
-        opkey = quiet? ? :deleteq : :delete
-        req = RequestFormatter.standard_request(opkey: opkey, key: key, cas: cas)
-        write(req)
-        @connection_manager.flush unless quiet?
-        response_processor.delete unless quiet?
-      end
-
-      # Pipelined delete - writes a quiet delete request without reading response.
-      # Used by PipelinedDeleter for bulk operations.
-      def pipelined_delete(key)
-        req = RequestFormatter.standard_request(opkey: :deleteq, key: key, cas: 0)
-        write(req)
-      end
-
-      # Arithmetic Commands
-      def decr(key, count, ttl, initial)
-        opkey = quiet? ? :decrq : :decr
-        decr_incr opkey, key, count, ttl, initial
-      end
-
-      def incr(key, count, ttl, initial)
-        opkey = quiet? ? :incrq : :incr
-        decr_incr opkey, key, count, ttl, initial
-      end
-
-      # This allows us to special case a nil initial value, and
-      # handle it differently than a zero.  This special value
-      # for expiry causes memcached to return a not found
-      # if the key doesn't already exist, rather than
-      # setting the initial value
-      NOT_FOUND_EXPIRY = 0xFFFFFFFF
-      private_constant :NOT_FOUND_EXPIRY
-
-      def decr_incr(opkey, key, count, ttl, initial)
-        expiry = initial ? TtlSanitizer.sanitize(ttl) : NOT_FOUND_EXPIRY
-        initial ||= 0
-        write(RequestFormatter.decr_incr_request(opkey: opkey, key: key,
-                                                 count: count, initial: initial, expiry: expiry))
-        @connection_manager.flush unless quiet?
-        response_processor.decr_incr unless quiet?
-      end
-
-      # Other Commands
-      def flush(ttl = 0)
-        opkey = quiet? ? :flushq : :flush
-        write(RequestFormatter.standard_request(opkey: opkey, ttl: ttl))
-        @connection_manager.flush unless quiet?
-        response_processor.no_body_response unless quiet?
-      end
-
-      # Noop is a keepalive operation but also used to demarcate the end of a set of pipelined commands.
-      # We need to read all the responses at once.
-      def noop
-        write_noop
-        response_processor.consume_all_responses_until_noop
-      end
-
-      def stats(info = '')
-        req = RequestFormatter.standard_request(opkey: :stat, key: info)
-        write(req)
-        @connection_manager.flush
-        response_processor.stats
-      end
-
-      def reset_stats
-        write(RequestFormatter.standard_request(opkey: :stat, key: 'reset'))
-        @connection_manager.flush
-        response_processor.reset
-      end
-
-      def version
-        write(RequestFormatter.standard_request(opkey: :version))
-        @connection_manager.flush
-        response_processor.version
-      end
-
-      def write_noop
-        req = RequestFormatter.standard_request(opkey: :noop)
-        write(req)
-        @connection_manager.flush
-      end
-
-      require_relative 'binary/request_formatter'
-      require_relative 'binary/response_header'
-      require_relative 'binary/response_processor'
-      require_relative 'binary/sasl_authentication'
-      include SaslAuthentication
-    end
-  end
-end

data/lib/dalli/protocol_deprecations.rb

@@ -1,45 +0,0 @@
-# frozen_string_literal: true
-
-module Dalli
-  ##
-  # Handles deprecation warnings for protocol and authentication features
-  # that will be removed in Dalli 5.0.
-  ##
-  module ProtocolDeprecations
-    BINARY_PROTOCOL_DEPRECATION_MESSAGE = <<~MSG.chomp
-      [DEPRECATION] The binary protocol is deprecated and will be removed in Dalli 5.0. \
-      Please use `protocol: :meta` instead. The meta protocol requires memcached 1.6+. \
-      See https://github.com/petergoldstein/dalli for migration details.
-    MSG
-
-    SASL_AUTH_DEPRECATION_MESSAGE = <<~MSG.chomp
-      [DEPRECATION] SASL authentication is deprecated and will be removed in Dalli 5.0. \
-      SASL is only supported by the binary protocol, which is being removed. \
-      Consider using network-level security (firewall rules, VPN) or memcached's TLS support instead.
-    MSG
-
-    private
-
-    def emit_deprecation_warnings
-      emit_binary_protocol_deprecation_warning
-      emit_sasl_auth_deprecation_warning
-    end
-
-    def emit_binary_protocol_deprecation_warning
-      protocol = @options[:protocol]
-      # Binary is used when protocol is nil, :binary, or 'binary'
-      return if protocol.to_s == 'meta'
-
-      warn BINARY_PROTOCOL_DEPRECATION_MESSAGE
-      Dalli.logger.warn(BINARY_PROTOCOL_DEPRECATION_MESSAGE)
-    end
-
-    def emit_sasl_auth_deprecation_warning
-      username = @options[:username] || ENV.fetch('MEMCACHE_USERNAME', nil)
-      return unless username
-
-      warn SASL_AUTH_DEPRECATION_MESSAGE
-      Dalli.logger.warn(SASL_AUTH_DEPRECATION_MESSAGE)
-    end
-  end
-end