dalli 5.0.11 → 5.0.13

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 43164df9e3e36d747346b72c44ebe1a7a3be0dee501c83b2ef813a39d14d9bb9
-  data.tar.gz: e8e1ecbf3d019891da3935c2d0c805f09bfc8a9746df14d9a6954f8cdecfdc10
+  metadata.gz: f386fb06afa43acffc1a4c63b7c611930d6302c90d10df138b8a0c24182fadcb
+  data.tar.gz: 5fd63314770f91b3f3668613e37f19fbe582c7e31ba1a5043155c5cd999b3560
 SHA512:
-  metadata.gz: e5b73ad3b6a075030a75f7f7903a097611cfd862f65a80a549d11f1c34c900205ac4ed4783392be4ee139c598b6de346ddaf782460c0ec11e8fc31847b4cb4aa
-  data.tar.gz: 5978a35d174e82c4cdc1dafdd439395f64035517f5a4592ee0ad1724e6b9b4241551c5b5990e6a6823d433e68604697bb43940e8bf3b82b15952325264e8b446
+  metadata.gz: 0d5368195322b97078321fe4385be067cd004e3d66dcf1e65329474d9161fcfeee3a227ec1988dbc693b27cd5bd91dfc6ab9332ee8a5509e321fa8baafe16b55
+  data.tar.gz: c18b9a743113cf553698a0238d9d773d498cd93a21f1593b798ff5432438a6be00b86ae049157026cc06541491117b19cead5c432777a4e372c902df5b17fa68

data/CHANGELOG.md

@@ -4,6 +4,38 @@ Dalli Changelog
 Unreleased
 ==========
 
+5.0.13
+==========
+
+Bug fixes:
+
+- `set_multi` across more than one server stored `nil` for a Symbol key or a key over 250 bytes, in place of the value given
+
+5.0.12
+==========
+
+Security:
+
+- Keep a connection shared by several threads or fibers in step during a `quiet` block (GHSA-5xvx-r8rg-5qm2)
+  - A `quiet` (or `multi`) block is quiet only for the thread or fiber that runs it. memcached still replies to a quiet request that fails (for example an `add` of a key that exists), and another thread or fiber using the same client while the block was open read that reply as its own, so its later requests could return another key's value. A non-quiet request now reads any pending quiet replies first
+- Don't let a `get_multi` that stopped early leave replies for a later request (GHSA-5xvx-r8rg-5qm2)
+  - A pipelined `get_multi` stopped by a non-network exception (its block raising or breaking, `Timeout.timeout`, or a value that can't be deserialized) left its unread replies in the response buffer, and a later `get_multi` large enough to use the interleaved path returned them as its own. The buffer is now cleared when a connection closes or a pipelined get starts, and unfinished pipelines are aborted
+  - Affects every supported line (the `get_multi` issue, 4.3 and later); fixed in 5.2.4, 5.1.6, 5.0.12, 4.3.9 and 3.2.15
+
+Bug fixes:
+
+- A per-request `raw: true` write no longer compresses the value (backport of #1213)
+  - Values at or above the compression threshold (4 KB by default) were compressed and flagged even with `raw: true`. Raw reads ignore flags, so reading them back with `raw: true` returned the compressed bytes
+  - Raw values are now stored exactly as given. Callers that pass `raw: true` without `compress: false` store large values uncompressed; Rails' `MemCacheStore` passes `compress: false` and isn't affected
+- Handle failed TLS handshakes like other connection failures, and warn when TLS verification is off (backport of #1215)
+  - A failed or timed-out handshake left the TCP socket open until GC and raised `OpenSSL::SSL::SSLError` directly, so the server was never marked down. The socket is now closed, and the error goes through the usual retry and `socket_max_failures` handling, raising `Dalli::NetworkError` (or `Dalli::RingError` once no server is available)
+  - Certificate and hostname checks come only from the `ssl_context` you pass, and a bare `OpenSSL::SSL::SSLContext.new` checks neither. Dalli now logs a warning once per process when `verify_mode` doesn't include `VERIFY_PEER` or `verify_hostname` is false; see the new TLS section in the README
+- A reply line longer than 8 KiB without a CRLF, or one cut short by the connection closing, is now a network error that closes the connection. Before, the line had no length limit (backport of #1216)
+- One value that can't be stored no longer discards other buffered writes (backport of #1218)
+  - A value that raised `Dalli::MarshalError` or `Dalli::ValueOverMaxSize` closed the connection, which dropped writes still buffered from the same `quiet` block or `set_multi`. Multi-server `set_multi` did this without raising
+  - Both `set_multi` paths now store every other key, skip the bad one, and then raise. Single-server `set_multi` used to store nothing
+  - When single-server `set_multi` failed to marshal a value, the error log included every value in the batch. It now lists only the keys
+
 5.0.11
 ==========
 

data/README.md

@@ -62,6 +62,25 @@ See the [5.0-Upgrade.md](5.0-Upgrade.md) guide for upgrade information.
 
 memcached names an item made by auto-create for a base64-encoded key (one with whitespace, control or non-ASCII characters) by its raw key bytes in later replies. Since 5.0.11, Dalli creates counters for such keys with an add rather than auto-create. Items created before then, or by other clients, can still be named that way until memcached releases a fix (1.6.45 and earlier behave this way). If keys passed to `incr` or `decr` with an initial value can contain untrusted input, set `verify_reply_keys: true` until then: multi-gets then ignore replies for keys they didn't request, at a cost of about 15% per multi-get.
 
+## TLS
+
+Pass an `OpenSSL::SSL::SSLContext` as `ssl_context:` to connect over TLS. Certificate and hostname verification come only from that context: Dalli adds no checks of its own. A bare `OpenSSL::SSL::SSLContext.new` verifies nothing, so it accepts any certificate from any server. Configure the context to verify both:
+
+```ruby
+ssl_context = OpenSSL::SSL::SSLContext.new
+ssl_context.set_params(
+  verify_mode: OpenSSL::SSL::VERIFY_PEER,
+  verify_hostname: true,
+  ca_file: '/path/to/ca.crt' # omit to use the system's trusted certificates
+)
+
+Dalli::Client.new('memcached.example.com:11211', ssl_context: ssl_context)
+```
+
+Dalli logs a warning once per process when the context's `verify_mode` is `VERIFY_NONE`, or when it is `VERIFY_PEER` but `verify_hostname` is false.
+
+A failed TLS handshake counts as a failed connection attempt, like a refused connection: once `socket_max_failures` attempts have failed, the server is marked down and requests fail over to the other servers.
+
 ## OpenTelemetry Tracing
 
 Dalli automatically instruments operations with [OpenTelemetry](https://opentelemetry.io/) when the SDK is present. No configuration is required - just add the OpenTelemetry gems to your application:

data/lib/dalli/client.rb

@@ -41,9 +41,10 @@ module Dalli
     #                 #fetch operations.
     # - :raw        - If set, disables serialization and compression entirely at the client level.
     #                 Only String values are supported. This is useful when the caller handles its own
-    #                 serialization (e.g., Rails' ActiveSupport::Cache). Note: this is different from
-    #                 the per-request :raw option which converts values to strings but still uses the
-    #                 serialization pipeline.
+    #                 serialization (e.g., Rails' ActiveSupport::Cache). The per-request :raw option
+    #                 (e.g. set(key, value, ttl, raw: true)) does the same for a single call: String
+    #                 values are stored as-is, uncompressed, and non-String values raise
+    #                 Dalli::MarshalError.
     # - :digest_class - defaults to Digest::MD5, allows you to pass in an object that responds to the hexdigest method,
     #                   useful for injecting a FIPS compliant hash object.
     # - :otel_db_statement - controls the +db.query.text+ span attribute when OpenTelemetry is loaded.
@@ -293,6 +294,11 @@ module Dalli
     # Any error replies will be discarded at the end of the block, and
     # Dalli client methods invoked inside the block will not
     # have return values
+    #
+    # The block is quiet only for the thread or fiber that runs it. Another
+    # thread or fiber sharing this client can make requests meanwhile; a
+    # non-quiet one first sends the block's buffered requests and discards
+    # their replies, so it reads only its own.
     def quiet
       old = Thread.current[::Dalli::QUIET]
       Thread.current[::Dalli::QUIET] = true

data/lib/dalli/options.rb

@@ -43,7 +43,7 @@ module Dalli
       end
     end
 
-    def pipeline_abort
+    def pipeline_abort(...)
       @lock.synchronize do
         super
       end

data/lib/dalli/pipelined_getter.rb

@@ -24,15 +24,21 @@ module Dalli
       return {} if keys.empty?
 
       @ring.lock do
-        # Stores partial results collected during interleaved send phase
-        @partial_results = {}
-        servers = setup_requests(keys)
-        start_time = Process.clock_gettime(Process::CLOCK_MONOTONIC)
+        # The servers with replies still to read, updated in place
+        servers = []
+        begin
+          # Stores partial results collected during interleaved send phase
+          @partial_results = {}
+          setup_requests(servers, keys)
+          start_time = Process.clock_gettime(Process::CLOCK_MONOTONIC)
 
-        # First yield any partial results collected during interleaved send
-        yield_partial_results(&block)
+          # First yield any partial results collected during interleaved send
+          yield_partial_results(&block)
 
-        servers = fetch_responses(servers, start_time, @ring.socket_timeout, &block) until servers.empty?
+          fetch_responses(servers, start_time, @ring.socket_timeout, &block) until servers.empty?
+        ensure
+          abort_unfinished(servers)
+        end
       end
     rescue Dalli::RetryableNetworkError => e
       Dalli.logger.debug { e.inspect }
@@ -49,12 +55,15 @@ module Dalli
       @partial_results.clear
     end
 
-    def setup_requests(keys)
+    # Adds the servers with a pending response to servers before writing
+    # their queries, so #process can abort them if it stops early.
+    def setup_requests(servers, keys)
       groups = groups_for_keys(keys)
+      servers.concat(groups.keys)
       make_getkq_requests(groups)
 
       # TODO: How does this exit on a NetworkError
-      finish_queries(groups.keys)
+      finish_queries(servers)
     end
 
     ##
@@ -116,29 +125,39 @@ module Dalli
       raise
     end
 
-    # Swallows Dalli::NetworkError
+    # Swallows Dalli::NetworkError. Empties servers, so they aren't aborted
+    # again.
     def abort_without_timeout(servers)
       servers.each(&:pipeline_abort)
+      servers.clear
+    end
+
+    # When #process stops before reading every server's replies (the caller's
+    # block raises or breaks, a Timeout.timeout fires, a value can't be
+    # deserialized), the rest are still on the socket and some may be in the
+    # response buffer. Closing those connections keeps them from being read
+    # as a later request's replies. Aborts that #process already handled
+    # emptied servers.
+    def abort_unfinished(servers)
+      servers.each { |server| server.pipeline_abort(timed_out: false) }
+      servers.clear
     end
 
+    # Reads the replies available, removing finished servers from servers
     def fetch_responses(servers, start_time, timeout, &block)
       # Remove any servers which are not connected
       servers.select!(&:connected?)
-      return [] if servers.empty?
+      return if servers.empty?
 
       time_left = remaining_time(start_time, timeout)
       readable_servers = servers_with_response(servers, time_left)
-      if readable_servers.empty?
-        abort_with_timeout(servers)
-        return []
-      end
+      return abort_with_timeout(servers) if readable_servers.empty?
 
       # Loop through the servers with responses, and
       # delete any from our list that are finished
       readable_servers.each do |server|
         servers.delete(server) if process_server(server, &block)
       end
-      servers
     rescue NetworkError
       # Abort and raise if we encountered a network error.  This triggers
       # a retry at the top level on RetryableNetworkError.
@@ -155,12 +174,10 @@ module Dalli
 
     # Swallows Dalli::NetworkError
     def abort_with_timeout(servers)
-      abort_without_timeout(servers)
       servers.each do |server|
         Dalli.logger.debug { "memcached at #{server.name} did not response within timeout" }
       end
-
-      true # Required to simplify caller
+      abort_without_timeout(servers)
     end
 
     # Processes responses from a server.  Returns true if there are no

data/lib/dalli/pipelined_setter.rb

@@ -16,6 +16,9 @@ module Dalli
     # Writes multiple key-value pairs to memcached.
     # Raises an error if any server is unavailable.
     #
+    # A value that can't be marshalled or is over value_max_bytes is skipped;
+    # the other keys are still stored, then the first such error is raised.
+    #
     # @param hash [Hash] key-value pairs to set
     # @param ttl [Integer] time-to-live in seconds
     # @param req_options [Hash] options passed to each set operation
@@ -24,10 +27,12 @@ module Dalli
     def process(hash, ttl, req_options)
       return if hash.empty?
 
+      @value_error = nil
       @ring.lock do
         servers = setup_requests(hash, ttl, req_options)
         finish_requests(servers)
       end
+      raise @value_error if @value_error
     rescue Dalli::RetryableNetworkError => e
       Dalli.logger.debug { e.inspect }
       Dalli.logger.debug { 'retrying pipelined sets because of network error' }
@@ -36,9 +41,12 @@ module Dalli
 
     private
 
+    # Values are looked up by each key as sent. The caller's key can't be
+    # rebuilt from it: a long key is digested, and a Symbol becomes a String.
     def setup_requests(hash, ttl, req_options)
-      groups = groups_for_keys(hash.keys)
-      make_set_requests(groups, hash, ttl, req_options)
+      pairs = hash.transform_keys { |k| @key_manager.validate_key(k.to_s) }
+      groups = groups_for_keys(pairs.keys)
+      make_set_requests(groups, pairs, ttl, req_options)
       groups.keys
     end
 
@@ -46,12 +54,14 @@ module Dalli
     # Loop through the server-grouped sets of keys, writing
     # the corresponding quiet set requests to the appropriate servers
     ##
-    def make_set_requests(groups, hash, ttl, req_options)
+    def make_set_requests(groups, pairs, ttl, req_options)
       groups.each do |server, keys_for_server|
         keys_for_server.each do |key|
-          original_key = @key_manager.key_without_namespace(key)
-          value = hash[original_key]
-          server.request(:pipelined_set, key, value, ttl, req_options)
+          server.request(:pipelined_set, key, pairs[key], ttl, req_options)
+        rescue Dalli::MarshalError, Dalli::ValueOverMaxSize => e
+          # Nothing was written for this key, so the server's other keys are
+          # unaffected; raised by #process once the batch is complete
+          @value_error ||= e
         rescue DalliError, NetworkError => e
           Dalli.logger.debug { e.inspect }
           Dalli.logger.debug { "unable to set key #{key} for server #{server.name}" }
@@ -72,8 +82,7 @@ module Dalli
     end
 
     def groups_for_keys(keys)
-      validated_keys = keys.map { |k| @key_manager.validate_key(k.to_s) }
-      groups = @ring.keys_grouped_by_server(validated_keys)
+      groups = @ring.keys_grouped_by_server(keys)
 
       if (unfound_keys = groups.delete(nil))
         Dalli.logger.debug do

data/lib/dalli/protocol/base.rb

@@ -51,15 +51,25 @@ module Dalli
         begin
           request_completed = false
           @connection_manager.start_request!
+          drain_pending_quiet_responses(opkey)
           response = send(opkey, *args)
+          # Quiet requests can leave replies on the socket; remember that so a
+          # later non-quiet request knows to read them first
+          @connection_manager.mark_quiet_responses_pending! if quiet? && opkey != :noop
 
           # pipelined_get/pipelined_get_interleaved emit query but don't read the response(s)
           @connection_manager.finish_request! unless %i[pipelined_get pipelined_get_interleaved].include?(opkey)
 
           request_completed = true
           response
-        rescue Dalli::MarshalError => e
-          log_marshal_err(args.first, e)
+        rescue Dalli::MarshalError, Dalli::ValueOverMaxSize => e
+          log_marshal_err(args.first, e) if e.is_a?(Dalli::MarshalError)
+          # Raised only while the connection is clean: a value is marshalled
+          # before any of its request is written, and write_multi_req raises
+          # only after reading every reply. Closing the connection would
+          # discard earlier quiet writes still waiting in the write buffer.
+          @connection_manager.abort_request!
+          request_completed = true
           raise
         rescue Dalli::DalliError
           raise
@@ -106,7 +116,7 @@ module Dalli
         verify_pipelined_state(:getkq)
         write_noop
         # Use ensure_ready instead of reset to preserve any data already buffered
-        # during interleaved pipelined get draining
+        # while this get_multi's interleaved requests were sent
         response_buffer.ensure_ready
       end
 
@@ -160,6 +170,10 @@ module Dalli
             end
           end
 
+          # A request to this server from inside the block closes the
+          # connection, and with it the rest of this get_multi's replies
+          raise_pipeline_closed unless response_buffer.in_progress?
+
           # Get the next response from the buffer
           status, cas, key, value = response_buffer.process_single_getk_response
         end
@@ -169,21 +183,24 @@ module Dalli
       # rubocop:enable Metrics/AbcSize, Metrics/CyclomaticComplexity, Metrics/PerceivedComplexity
 
       # Abort current pipelined get. Generally used to signal an external
-      # timeout during pipelined get.  The underlying socket is
+      # timeout during pipelined get, or that the get_multi stopped before
+      # reading all its replies (timed_out: false).  The underlying socket is
       # disconnected, and the exception is swallowed.
       #
       # Returns nothing.
-      def pipeline_abort
+      def pipeline_abort(timed_out: true)
         response_buffer.clear
+        @pipeline_out_of_step = false
+        @expected_reply_keys = nil
         @connection_manager.abort_request!
         return true unless connected?
 
         # Closes the connection, which ensures that our connection is in a
         # clean state for future requests. The get_multi ran out of time
-        # overall rather than hitting a socket error, so it doesn't count
-        # toward socket_max_failures: two slow get_multis in a row would
-        # otherwise mark a healthy server down.
-        Dalli.logger.warn { "#{name} didn't finish a get_multi within socket_timeout; closing the connection" }
+        # overall, or stopped early, rather than hitting a socket error, so it
+        # doesn't count toward socket_max_failures: two slow get_multis in a
+        # row would otherwise mark a healthy server down.
+        log_pipeline_abort(timed_out)
         @connection_manager.close
         true
       end
@@ -215,6 +232,18 @@ module Dalli
       ALLOWED_QUIET_OPS = %i[add replace set delete incr decr append prepend flush noop].freeze
       private_constant :ALLOWED_QUIET_OPS
 
+      # Reads the replies still pending from quiet requests before any
+      # non-quiet request, which reads a reply of its own. A quiet block reads
+      # them at its end, but the connection is shared: another thread or fiber
+      # (quiet? is fiber-local) can make a request while the block is still
+      # open, and would otherwise read the block's replies as its own. The
+      # noop also sends any quiet requests still buffered.
+      def drain_pending_quiet_responses(opkey)
+        return unless @connection_manager.quiet_responses_pending? && opkey != :noop && !quiet?
+
+        noop
+      end
+
       def verify_allowed_quiet!(opkey)
         return if ALLOWED_QUIET_OPS.include?(opkey)
 
@@ -289,8 +318,9 @@ module Dalli
       # This prevents socket buffer deadlock when sending many keys.
       # Populates the provided results hash with any responses drained during send.
       def pipelined_get_interleaved(keys, chunk_size, results)
-        # Initialize the response buffer for draining during send phase
-        response_buffer.ensure_ready
+        # Start with an empty buffer for draining during the send phase. Bytes
+        # left by an earlier get_multi that stopped early belong to its replies.
+        response_buffer.reset
         expect_reply_keys(keys)
 
         keys.each_slice(chunk_size) do |chunk|
@@ -337,8 +367,12 @@ module Dalli
         nil
       end
 
+      # Registered with the connection manager, which clears it whenever the
+      # connection closes, so a new connection never parses an old one's bytes
       def response_buffer
-        @response_buffer ||= ResponseBuffer.new(@connection_manager, response_processor)
+        @response_buffer ||= ResponseBuffer.new(@connection_manager, response_processor).tap do |buffer|
+          @connection_manager.response_buffer = buffer
+        end
       end
 
       # A multi-get only accepts replies for the keys it asked for. memcached
@@ -390,7 +424,23 @@ module Dalli
         @connection_manager.reconnect! 'pipelined get has completed' if pipeline_complete?
       end
 
+      def raise_pipeline_closed
+        raise Dalli::DalliError, "The connection to #{name} closed while a get_multi was reading its replies " \
+                                 '(a request made from inside the get_multi block?)'
+      end
+
+      def log_pipeline_abort(timed_out)
+        if timed_out
+          Dalli.logger.warn { "#{name} didn't finish a get_multi within socket_timeout; closing the connection" }
+        else
+          Dalli.logger.debug { "#{name} get_multi stopped before reading all replies; closing the connection" }
+        end
+      end
+
+      # The single-server set_multi path passes its whole pairs hash as the
+      # first argument; log only its keys, never the values
       def log_marshal_err(key, err)
+        key = key.keys.join("', '") if key.is_a?(Hash)
         Dalli.logger.error "Marshalling error for key '#{key}': #{err.message}"
         Dalli.logger.error 'You are trying to cache a Ruby object which cannot be serialized to memcached.'
       end

data/lib/dalli/protocol/connection_manager.rb

@@ -26,6 +26,9 @@ module Dalli
 
       attr_accessor :hostname, :port, :socket_type, :options
       attr_reader :sock
+      # The buffer replies are read into (see Base#response_buffer). It's
+      # cleared with the write buffer whenever the connection closes.
+      attr_writer :response_buffer
 
       def initialize(hostname, port, socket_type, client_options)
         @hostname = hostname
@@ -33,10 +36,12 @@ module Dalli
         @socket_type = socket_type
         @options = DEFAULTS.merge(client_options)
         @request_in_progress = false
+        @quiet_responses_pending = false
         @sock = nil
         @pid = nil
         @write_buffer = []
         @write_buffer_bytes = 0
+        @response_buffer = nil
 
         @fail_count = 0
         reset_down_info
@@ -58,8 +63,9 @@ module Dalli
         @sock.sync = true
         @pid = Process.pid
         @request_in_progress = false
-      rescue SystemCallError, *TIMEOUT_ERRORS, EOFError, SocketError => e
-        # SocketError = DNS resolution failure
+        @quiet_responses_pending = false
+      rescue SystemCallError, *TIMEOUT_ERRORS, *SSL_ERRORS, EOFError, SocketError => e
+        # SocketError = DNS resolution failure; SSL_ERRORS = failed TLS handshake
         error_on_request!(e)
       end
 
@@ -129,7 +135,9 @@ module Dalli
           # @request_in_progress == true.
           @sock = nil
           @pid = nil
+          @quiet_responses_pending = false
           discard_write_buffer
+          @response_buffer&.clear
           abort_request!
         end
       end
@@ -145,6 +153,21 @@ module Dalli
         !@sock.nil?
       end
 
+      # True when a quiet request was written on this connection and its
+      # replies (memcached only suppresses some of them) haven't been
+      # consumed by a noop yet.
+      def quiet_responses_pending?
+        @quiet_responses_pending
+      end
+
+      def mark_quiet_responses_pending!
+        @quiet_responses_pending = true
+      end
+
+      def clear_quiet_responses_pending!
+        @quiet_responses_pending = false
+      end
+
       def request_in_progress?
         @request_in_progress
       end
@@ -170,10 +193,22 @@ module Dalli
         @request_in_progress = false
       end
 
+      # The longest reply line Dalli reads, in bytes, CRLF included. A meta
+      # reply header is at most about 500 bytes: "VA <size>" plus its flags,
+      # the largest being a 250-byte key base64-encoded to 336 bytes, a
+      # 20-digit CAS, a 32-byte opaque and a few numeric fields. The longest
+      # stats line (stats settings) carries a file path such as ext_path, up
+      # to PATH_MAX (4096). 8 KiB leaves room for both, while a server that
+      # never sends CRLF can't grow the read buffer without bound.
+      MAX_LINE_BYTES = 8 * 1024
+
       def read_line
         flush_write_buffer
-        data = @sock.gets("\r\n")
+        data = @sock.gets("\r\n", MAX_LINE_BYTES)
         error_on_request!('EOF in read_line') if data.nil?
+        # Without its CRLF the line either ran past MAX_LINE_BYTES or was cut
+        # short by EOF; either way the connection is out of step
+        error_on_request!("Reply line without CRLF in the first #{MAX_LINE_BYTES} bytes") unless data.end_with?("\r\n")
         data
       rescue SystemCallError, *TIMEOUT_ERRORS, *SSL_ERRORS, EOFError => e
         error_on_request!(e)

data/lib/dalli/protocol/meta.rb

@@ -242,7 +242,10 @@ module Dalli
       # We need to read all the responses at once.
       def noop
         write_noop
-        response_processor.consume_all_responses_until_mn
+        result = response_processor.consume_all_responses_until_mn
+        # Everything up to and including the MN has been read
+        @connection_manager.clear_quiet_responses_pending!
+        result
       end
 
       def stats(info = nil)
@@ -313,16 +316,27 @@ module Dalli
 
       # Single-server fast path for set_multi. Inlines request formatting to
       # minimize per-key overhead. Avoids PipelinedSetter server grouping.
+      #
+      # A value that can't be marshalled or is over value_max_bytes is skipped;
+      # the other keys are still stored, then the first such error is raised
+      # (as PipelinedSetter does on the multi-server path).
       def write_multi_req(pairs, ttl, req_options)
         ttl = TtlSanitizer.sanitize(ttl) if ttl
-        entries = pairs.map do |key, raw_value|
+        value_error = nil
+        entries = pairs.filter_map do |key, raw_value|
           [key, @value_marshaller.store(key, raw_value, req_options)]
+        rescue Dalli::MarshalError, Dalli::ValueOverMaxSize => e
+          value_error ||= e
+          nil
         end
 
-        buffer = RequestFormatter.multi_meta_set(entries, ttl: ttl)
-        flushed_write(buffer)
-        buffer.clear
-        response_processor.consume_all_responses_until_mn
+        unless entries.empty?
+          buffer = RequestFormatter.multi_meta_set(entries, ttl: ttl)
+          flushed_write(buffer)
+          buffer.clear
+          response_processor.consume_all_responses_until_mn
+        end
+        raise value_error if value_error
       end
 
       # Single-server fast path for delete_multi. Writes all quiet delete requests

data/lib/dalli/protocol/value_compressor.rb

@@ -84,11 +84,13 @@ module Dalli
       end
 
       # Checks whether we should apply compression when serializing a value
-      # based on the specified options.  Returns false unless the value
-      # is greater than the minimum compression size.  Otherwise returns
-      # based on a method-level option if specified, falling back to the
-      # server default.
+      # based on the specified options.  Returns false for a raw request
+      # (raw reads ignore bitflags, so the value must be stored as given) or
+      # unless the value is greater than the minimum compression size.
+      # Otherwise returns based on a method-level option if specified,
+      # falling back to the server default.
       def compress_value?(value, req_options)
+        return false if req_options&.dig(:raw)
         return false unless value.bytesize >= compression_min_size
         return compress_by_default? unless req_options && !req_options[:compress].nil?
 

data/lib/dalli/protocol.rb

@@ -1,5 +1,6 @@
 # frozen_string_literal: true
 
+require 'openssl'
 require 'timeout'
 
 module Dalli
@@ -16,14 +17,11 @@ module Dalli
         [Timeout::Error]
       end
 
-    # SSL errors that occur during read/write operations (not during initial
-    # handshake) should trigger reconnection. These indicate transient network
-    # issues, not configuration problems.
-    SSL_ERRORS =
-      if defined?(OpenSSL::SSL::SSLError)
-        [OpenSSL::SSL::SSLError]
-      else
-        []
-      end
+    # SSL errors during the handshake or during read/write operations count as
+    # a failed request: they trigger reconnection and, after
+    # socket_max_failures, mark the server down.
+    # openssl is required above: this file loads before dalli/socket, so a
+    # defined? check here would find OpenSSL missing and leave this empty.
+    SSL_ERRORS = [OpenSSL::SSL::SSLError].freeze
   end
 end

data/lib/dalli/socket.rb

@@ -203,12 +203,41 @@ module Dalli
         [seconds, microseconds].pack(timeval_pack_format(sock))
       end
 
+      # Closes the TCP socket if the handshake fails or times out, so a failed
+      # attempt doesn't hold a file descriptor until GC.
       def self.wrapping_ssl_socket(tcp_socket, host, ssl_context)
+        warn_if_unverified(ssl_context)
+        connected = false
         ssl_socket = Dalli::Socket::SSLSocket.new(tcp_socket, ssl_context)
         ssl_socket.hostname = host
         ssl_socket.sync_close = true
         ssl_socket.connect
+        connected = true
         ssl_socket
+      ensure
+        tcp_socket.close unless connected
+      end
+
+      # Class variable to track whether the TLS verification warning has been logged
+      @@ssl_verification_warning_logged = false # rubocop:disable Style/ClassVars
+
+      # Verification comes entirely from the caller's ssl_context, and a bare
+      # SSLContext.new verifies nothing, so warn (once per process) when the
+      # certificate or the hostname won't be checked.
+      def self.warn_if_unverified(ssl_context)
+        return if @@ssl_verification_warning_logged
+
+        problem = if !ssl_context.verify_mode.to_i.anybits?(::OpenSSL::SSL::VERIFY_PEER)
+                    'verify_mode is VERIFY_NONE, so the server certificate is not checked'
+                  elsif !ssl_context.verify_hostname
+                    'verify_hostname is false, so the certificate is not checked against the hostname'
+                  end
+        return unless problem
+
+        Dalli.logger.warn "SECURITY WARNING: Dalli's ssl_context #{problem}. " \
+                          'Use ssl_context.set_params(verify_mode: OpenSSL::SSL::VERIFY_PEER, ' \
+                          'verify_hostname: true) to verify the server.'
+        @@ssl_verification_warning_logged = true # rubocop:disable Style/ClassVars
       end
     end
 

data/lib/dalli/version.rb

@@ -1,7 +1,7 @@
 # frozen_string_literal: true
 
 module Dalli
-  VERSION = '5.0.11'
+  VERSION = '5.0.13'
 
   MIN_SUPPORTED_MEMCACHED_VERSION = '1.6'
 end

metadata

@@ -1,7 +1,7 @@
 --- !ruby/object:Gem::Specification
 name: dalli
 version: !ruby/object:Gem::Version
-  version: 5.0.11
+  version: 5.0.13
 platform: ruby
 authors:
 - Peter M. Goldstein