dalli 5.1.2 → 5.1.4

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 53113b610a737a8a40a4f830c24251218161e36a83c01e3cf92b85e74ce2bbf1
-  data.tar.gz: 510c940cb0ee92a95e77939ffe277fc1a299539dff69d7e70cb2a8ca621ae94f
+  metadata.gz: 70f17667b9c623f9233be342d920de385bab797b44a4bc9f1e85c233f766404f
+  data.tar.gz: 527b9ff3dd4699ad5d307107668b31d3a739675825ad4d0b2cfb35ac09581a81
 SHA512:
-  metadata.gz: eacfdd099e2236d46abc8c445467f34088a23455cf42fa9cef9a328dfc64b474c0456063c2b40fb7e958ae96a455c63bf29ea8ad118501f64a46090364999d68
-  data.tar.gz: c0327598c3d0384e50bc132ef44cd5ccb94b346969eeee4b2235d43c7e31f2cde4fc9867b81c8f5ceffb6c624a01a1b3ea8de44a6ac856521fbcaa714d2a516d
+  metadata.gz: 9e54f454a1bca3b79f229a219b458533d2778c1521043c1ee7c2d203ae0d25e1449fc375b59754aaadac79de6ea484170ba52c10008da6abd40451821300a60c
+  data.tar.gz: 1dea61992e48d8786dbb3ef929e30c53d3544bafed895334cf4316ad0710791c4ebb0887fb4fa63b79358c0029be11b3495a2875a24956a296246fe9d5e4eaa0

data/CHANGELOG.md

@@ -4,6 +4,65 @@ Dalli Changelog
 Unreleased
 ==========
 
+5.1.4
+==========
+
+Security:
+
+- Keep the caller's key when retrying a request (GHSA-m252-9cgf-vx2w)
+  - With a `namespace`, a request retried after a transient network error (a timeout, or a connection closed by memcached or a proxy) applied the namespace a second time, so a retried read could return a different key's value and a retried write could overwrite a different key
+  - Affects single-key operations since 5.0.3, single-server `get_multi` since 5.1.0, and `get_with_metadata` and `fetch_with_lock` since 4.1.0; fixed in 5.2.2, 5.1.4, 5.0.10 and 4.3.7. Clients without a namespace aren't affected
+- Complete the fix for per-request `raw: true` on reads (GHSA-wr87-m4jw-29x5)
+  - A read made with `raw: true` doesn't ask for flags, but a reply carrying them anyway (from a proxy or a hostile server) still had its value deserialized. Raw reads now ignore flags in the reply
+  - Affects 5.1.3 and earlier; fixed in 5.2.2, 5.1.4, 5.0.10, 4.3.7 and 3.2.13
+
+Bug fixes:
+
+- `get_with_metadata` and `fetch_with_lock` retried the final error raised when a server is marked down, so with `down_retry_delay: 0` they retried an unresponsive server forever. They now retry only retryable errors, like other operations
+- Fix a regression in 5.1.3: a `get_multi` that didn't finish within `socket_timeout` counted toward `socket_max_failures`, so two slow `get_multi` calls in a row marked a healthy server down. It now just closes the connection
+- Fix a regression in 5.1.3: over TLS, every request buffered in a `quiet` block was sent as its own TLS record and system call (2.5 times slower for a block of 2000 deletes). Each flush is one write again
+- Handle malformed replies from a broken or hostile server or proxy: a negative value size, a hit with no key, or a hit with no `s` flag could leave the connection out of step or raise a non-Dalli error
+- With `decompressed_max_bytes` in effect, data after the end of a compressed value's stream was returned as part of the value. It's now ignored, as without the limit
+- With a `digest_class` whose digests aren't short hex strings, shortening a long key could loop forever. It now raises `ArgumentError`
+
+Development:
+
+- Run the Tests, RuboCop and Profiles workflows on pushes to `main` and the `*-stable` branches only, so a pull request's branch isn't tested twice (backport of #1198)
+- Fix flaky failover tests: move their ports out of Linux's ephemeral port range, and wait for memcached to accept connections after starting it (backport of #1184)
+
+5.1.3
+==========
+
+Security:
+
+- Fix pipelined `get_multi` returning one key's value for another after an error reply (GHSA-p6pm-ch9v-44vx)
+  - The pipelined reply parser took any reply without a body for the end of the batch. An error reply for one key (`CLIENT_ERROR`, `SERVER_ERROR`, or `EN` from a proxy) ended it early, and the replies still on the connection were read as the replies to later commands, so a `get` could return another key's value
+  - Keys are now measured in bytes as sent, after base64 encoding when the key needs it. A key under 250 characters but over 250 bytes on the wire (for example, one with many non-ASCII characters) was the easiest way to cause that error reply. Such keys are now truncated like other long keys; keys that worked before are unchanged
+  - Affects the meta protocol since 3.2.0 (the default since 5.0.0); fixed in 5.2.1, 5.1.3, 5.0.9, 4.3.6 and 3.2.12
+- Reject routing tokens that add meta flags, and stop retrying an unresponsive server forever (GHSA-4qp6-2jcr-596v)
+  - `p_token` and `l_token` now reject whitespace and control characters, not just CR, LF and NUL. A space let a token add meta flags to the request it was sent with: changing an item's TTL on a read, creating stub items on a miss, turning a delete into a stale tombstone, or reading a different key
+  - Affects 5.1.0 and later; fixed in 5.2.1 and 5.1.3
+  - A server that accepted connections but never answered (or dropped the connection on a request) was retried forever, hanging the caller, because each successful reconnect reset the failure count. Requests now fail after `socket_max_failures` attempts and the server is marked down, as when it can't be reached at all; it gets a full set of attempts again after `down_retry_delay`
+  - Affects all versions; fixed in 5.2.1, 5.1.3, 5.0.9, 4.3.6 and 3.2.12
+- Limit decompressed and reply value sizes (GHSA-3553-vcg5-72jw)
+  - Values flagged as compressed were inflated without limit, so a small stored value could expand to gigabytes on read, whatever the client's `compress` or `serializer` settings. The new `decompressed_max_bytes` option (default 128 MiB; `nil` disables it) makes a read that would pass it raise `Dalli::UnmarshalError`. Custom compressors whose `decompress` takes only the data keep working, without the limit
+  - The value size in a reply was used to read or buffer that many bytes, so a malicious server could make the client allocate gigabytes. Sizes over 1 GiB, memcached's largest item, now raise `Dalli::DalliError` before reading
+  - Affects all versions; fixed in 5.2.1, 5.1.3, 5.0.9, 4.3.6 and 3.2.12
+- Honor per-request `raw: true` on reads, and add `Dalli::JSONSerializer` (GHSA-wr87-m4jw-29x5)
+  - `get_multi`, `get_multi_cas`, `get_multi_with_metadata`, `get_cas` and `get_with_metadata` ignored a per-request `raw: true` and deserialized the value according to its stored flags, so a caller who asked for raw bytes could still have `Marshal.load` run on data someone else wrote to memcached. They now return the stored bytes
+  - The `raw` part affects `get_with_metadata` since 4.2.0, and the other methods since they gained per-request options in 5.1.0
+  - `serializer: JSON` reads values with `JSON.load`, which on json gem versions before 3.0 creates an object of the class named in a stored `json_class` key when the json additions are loaded. The new `Dalli::JSONSerializer` reads with `JSON.parse` and only returns plain JSON types. The README and the Marshal security warning now recommend it; `serializer: JSON` itself is unchanged
+  - The `serializer: JSON` part affects any version used with the json gem before 3.0
+- Keep a forked child from writing to or closing the parent's connection (GHSA-w39f-xq2m-4g8x)
+  - A child forked while the parent had requests buffered but not yet sent, such as quiet writes inside a `quiet` block, sent them again when it closed the client or reconnected: quiet writes ran twice, and the parent's later replies could be read as the replies to other requests, returning one key's value for another. Dalli now buffers requests itself and a forked child discards them
+  - Affects 4.2.0 and later; fixed in 5.2.1, 5.1.3, 5.0.9 and 4.3.6
+  - With TLS, a forked child that closed the client (or reconnected after detecting the fork) sent a TLS close on the connection the parent was still using, ending the parent's session. A forked child now closes only its own copy of the socket
+  - Affects all versions with TLS; fixed in 5.2.1, 5.1.3, 5.0.9, 4.3.6 and 3.2.12
+
+Bug fixes:
+
+- An exception raised by a `get_multi` block (such as `Timeout::Error`) was treated as a network failure: swallowed, with the whole `get_multi` retried and keys yielded twice. It now reaches the caller
+
 5.1.2
 ==========
 
@@ -16,6 +75,10 @@ Bug fixes:
   - Affects the meta protocol since 3.2.0 (the default since 5.0.0)
   - Thanks to Julian Richard Contreras for this contribution
 
+Development:
+
+- Add property tests for request building and fuzz tests for the reply parsers
+
 5.1.1
 ==========
 

data/README.md

@@ -54,9 +54,13 @@ The separator must be a single non-alphanumeric character. Valid examples: `:`,
 By default, Dalli uses Ruby's Marshal for serialization. Deserializing untrusted data with Marshal can lead to remote code execution. If you cache user-controlled data, consider using a safer serializer:
 
 ```ruby
-Dalli::Client.new('localhost:11211', serializer: JSON)
+Dalli::Client.new('localhost:11211', serializer: Dalli::JSONSerializer)
 ```
 
+`Dalli::JSONSerializer` reads values with `JSON.parse`, so it only ever returns plain hashes, arrays, strings, numbers, booleans and `nil`. Passing the `JSON` module itself (`serializer: JSON`) reads values with `JSON.load`, which can create objects of other classes from a stored `json_class` key and is not recommended for data an attacker might write.
+
+Per-request `raw: true` returns the stored bytes without deserializing or decompressing them, on every read method (`get`, `get_multi`, `get_cas`, `get_with_metadata` and `fetch`).
+
 See the [5.0-Upgrade.md](5.0-Upgrade.md) guide for upgrade information.
 
 ## OpenTelemetry Tracing

data/lib/dalli/client.rb

@@ -136,14 +136,18 @@ module Dalli
     #
     def get_with_metadata(key, options = {})
       validate_routing_tokens!(options)
-      key = key.to_s
-      key = @key_manager.validate_key(key)
+      # A retry re-runs this method with its original arguments, so `key` must
+      # stay as the caller gave it; reassigning it would add the namespace again.
+      validated_key = @key_manager.validate_key(key.to_s)
 
-      server = ring.server_for_key(key)
-      Instrumentation.trace('get_with_metadata', trace_attrs('get_with_metadata', key, server)) do
-        server.request(:meta_get, key, options)
+      server = ring.server_for_key(validated_key)
+      Instrumentation.trace('get_with_metadata', trace_attrs('get_with_metadata', validated_key, server)) do
+        server.request(:meta_get, validated_key, options)
       end
-    rescue NetworkError => e
+    # Only a retryable error is retried. The NetworkError raised when a server
+    # is marked down is final, as in #perform; retrying it looped forever once
+    # down_retry_delay let the same server be tried again.
+    rescue RetryableNetworkError => e
       Dalli.logger.debug { e.inspect }
       Dalli.logger.debug { 'retrying get_with_metadata with new server' }
       retry
@@ -302,14 +306,17 @@ module Dalli
       validate_integer!(:lock_ttl, lock_ttl)
       validate_integer!(:recache_threshold, recache_threshold)
       validate_routing_tokens!(req_options)
-      key = key.to_s
-      key = @key_manager.validate_key(key)
+      # Left unchanged for the retry below; see get_with_metadata
+      validated_key = @key_manager.validate_key(key.to_s)
 
-      server = ring.server_for_key(key)
-      Instrumentation.trace('fetch_with_lock', trace_attrs('fetch_with_lock', key, server)) do
-        fetch_with_lock_request(key, ttl, lock_ttl, recache_threshold, req_options, &block)
+      server = ring.server_for_key(validated_key)
+      Instrumentation.trace('fetch_with_lock', trace_attrs('fetch_with_lock', validated_key, server)) do
+        fetch_with_lock_request(validated_key, ttl, lock_ttl, recache_threshold, req_options, &block)
       end
-    rescue NetworkError => e
+    # Only a retryable error is retried. The NetworkError raised when a server
+    # is marked down is final, as in #perform; retrying it looped forever once
+    # down_retry_delay let the same server be tried again.
+    rescue RetryableNetworkError => e
       Dalli.logger.debug { e.inspect }
       Dalli.logger.debug { 'retrying fetch_with_lock with new server' }
       retry
@@ -727,10 +734,11 @@ module Dalli
     # to route to at all is still silent, matching Ring#keys_grouped_by_server
     # dropping a key it can't route on both the single- and multi-server paths.
     def single_server_get_multi(keys, req_options = nil)
-      keys.map! { |k| @key_manager.validate_key(k.to_s) }
+      # A new array, so the retry below starts again from the caller's keys
+      validated_keys = keys.map { |k| @key_manager.validate_key(k.to_s) }
       return {} unless (server = single_server)
 
-      result = server.request(:read_multi_req, keys, req_options)
+      result = server.request(:read_multi_req, validated_keys, req_options)
       result.transform_keys! { |k| @key_manager.key_without_namespace(k) }
       result
     rescue Dalli::RetryableNetworkError => e
@@ -821,7 +829,7 @@ module Dalli
     # check means unwinding through Protocol::Base#request, which logs the
     # failure as unexpected and closes the connection -- a caller passing a
     # bad token should get a clean ArgumentError and keep its connection.
-    ROUTING_TOKEN_FORBIDDEN = /[\r\n\0]/
+    ROUTING_TOKEN_FORBIDDEN = /[\x00-\x20\x7F]/
     private_constant :ROUTING_TOKEN_FORBIDDEN
 
     def validate_routing_tokens!(req_options)
@@ -837,7 +845,9 @@ module Dalli
       # here (it would also excuse [] / {} from the type check below).
       return if value.nil? || (value.is_a?(String) && value.empty?)
       raise ArgumentError, "#{name} must be a String, got #{value.class}" unless value.is_a?(String)
-      raise ArgumentError, "#{name} must not contain CRLF or null bytes" if value.match?(ROUTING_TOKEN_FORBIDDEN)
+      return unless value.match?(ROUTING_TOKEN_FORBIDDEN)
+
+      raise ArgumentError, "#{name} must not contain whitespace or control characters"
     end
 
     def cas_core(key, always_set, ttl = nil, req_options = nil)
@@ -895,18 +905,20 @@ module Dalli
       # rubocop:enable Naming/MethodParameterName
       return yield if block_given?
 
-      key = key.to_s
-      key = @key_manager.validate_key(key)
+      # `retry` below re-runs this method with its original arguments, so `key`
+      # must stay as the caller gave it: reassigning it would apply the
+      # namespace a second time and send the retry to a different key.
+      validated_key = @key_manager.validate_key(key.to_s)
 
-      server = ring.server_for_key(key)
+      server = ring.server_for_key(validated_key)
 
       if Instrumentation.enabled?
         op_name = op.name
-        Instrumentation.trace(op_name, trace_attrs(op_name, key, server)) do
-          server.request(op, key, ...)
+        Instrumentation.trace(op_name, trace_attrs(op_name, validated_key, server)) do
+          server.request(op, validated_key, ...)
         end
       else
-        server.request(op, key, ...)
+        server.request(op, validated_key, ...)
       end
     rescue RetryableNetworkError => e
       Dalli.logger.debug { e.inspect }

data/lib/dalli/compressor.rb

@@ -13,8 +13,54 @@ module Dalli
       Zlib::Deflate.deflate(data)
     end
 
-    def self.decompress(data)
-      Zlib::Inflate.inflate(data)
+    # DEFLATE can't expand data by more than about 1032:1
+    MAX_EXPANSION = 1032
+    # Compressed input is fed to zlib this many bytes at a time. MRI's zlib
+    # yields the output in small chunks; one that returns it instead still
+    # produces at most MAX_EXPANSION times this per step.
+    INFLATE_SLICE_BYTES = 64 * 1024
+
+    # max_bytes caps the decompressed size; inflating stops and raises as soon
+    # as the output passes it, so a small compressed value can't expand into
+    # gigabytes of memory.
+    def self.decompress(data, max_bytes: nil)
+      return Zlib::Inflate.inflate(data) unless may_exceed?(data, max_bytes)
+
+      inflate_within_limit(Zlib::Inflate.new, data, max_bytes)
+    end
+
+    # False when even the largest possible expansion stays within max_bytes,
+    # so the value can be inflated in one step
+    def self.may_exceed?(data, max_bytes)
+      max_bytes ? data.bytesize * MAX_EXPANSION > max_bytes : false
+    end
+
+    def self.inflate_within_limit(inflater, data, max_bytes)
+      out = String.new(encoding: Encoding::BINARY)
+      0.step(data.bytesize - 1, INFLATE_SLICE_BYTES) do |pos|
+        inflate_slice(inflater, data.byteslice(pos, INFLATE_SLICE_BYTES), out, max_bytes)
+        # Anything after the end of the stream isn't part of the value, as
+        # with Zlib::Inflate.inflate; finish would return it as output
+        break if inflater.finished?
+      end
+      return out if inflater.finished?
+
+      append_within_limit(out, inflater.finish, max_bytes)
+    ensure
+      inflater.close
+    end
+
+    # Both yielded and returned output count toward the limit
+    def self.inflate_slice(inflater, slice, out, max_bytes)
+      rest = inflater.inflate(slice) { |chunk| append_within_limit(out, chunk, max_bytes) }
+      append_within_limit(out, rest, max_bytes)
+    end
+
+    def self.append_within_limit(out, chunk, max_bytes)
+      out << chunk if chunk
+      return out if out.bytesize <= max_bytes
+
+      raise Dalli::UnmarshalError, "Decompressed value exceeds #{max_bytes} bytes (decompressed_max_bytes)"
     end
   end
 
@@ -32,9 +78,11 @@ module Dalli
       io.string
     end
 
-    def self.decompress(data)
-      io = StringIO.new(data, 'rb')
-      Zlib::GzipReader.new(io).read
+    def self.decompress(data, max_bytes: nil)
+      return Zlib::GzipReader.new(StringIO.new(data, 'rb')).read unless Compressor.may_exceed?(data, max_bytes)
+
+      # MAX_WBITS + 16 makes zlib read the gzip format (and check its CRC)
+      Compressor.inflate_within_limit(Zlib::Inflate.new(Zlib::MAX_WBITS + 16), data, max_bytes)
     end
   end
 end

data/lib/dalli/json_serializer.rb

@@ -0,0 +1,28 @@
+# frozen_string_literal: true
+
+require 'json'
+
+module Dalli
+  ##
+  # A serializer that stores values as JSON and never creates Ruby objects
+  # from what it reads.
+  #
+  # Passing the JSON module itself as the serializer (serializer: JSON) reads
+  # values back with JSON.load, which on json gem versions before 3.0 honors
+  # "json_class" and instantiates other classes when the json gem's additions
+  # are loaded. This serializer reads with JSON.parse, so a value written to
+  # memcached by someone else can only come back as plain hashes, arrays,
+  # strings, numbers, booleans and nil.
+  #
+  #   Dalli::Client.new(servers, serializer: Dalli::JSONSerializer)
+  ##
+  module JSONSerializer
+    def self.dump(value)
+      JSON.generate(value)
+    end
+
+    def self.load(data)
+      JSON.parse(data)
+    end
+  end
+end

data/lib/dalli/key_manager.rb

@@ -58,7 +58,7 @@ module Dalli
       raise ArgumentError, 'key cannot be blank' unless key&.length&.positive?
 
       key = key_with_namespace(key)
-      key.length > MAX_KEY_LENGTH ? truncated_key(key) : key
+      wire_length(key) > MAX_KEY_LENGTH ? truncated_key(key) : key
     end
 
     ##
@@ -122,7 +122,28 @@ module Dalli
     ##
     def truncated_key(key)
       digest = digest_class.hexdigest(key)
-      "#{key[0, prefix_length(digest)]}#{TRUNCATED_KEY_SEPARATOR}#{digest}"
+      prefix = key[0, [prefix_length(digest), 0].max]
+      truncated = "#{prefix}#{TRUNCATED_KEY_SEPARATOR}#{digest}"
+      # A prefix with whitespace or non-ASCII characters sends the whole key
+      # base64-encoded, which can still be too long on the wire; shorten it.
+      while wire_length(truncated) > MAX_KEY_LENGTH && !prefix.empty?
+        prefix = prefix[0...-1]
+        truncated = "#{prefix}#{TRUNCATED_KEY_SEPARATOR}#{digest}"
+      end
+      return truncated if wire_length(truncated) <= MAX_KEY_LENGTH
+
+      # Only possible with a digest_class whose digests aren't short hex strings
+      raise ArgumentError, "#{digest_class} digests are too long to fit in a truncated key"
+    end
+
+    # memcached's key limit applies to the key as sent: in bytes, and after
+    # base64 encoding when the key needs it. A key can be within the limit in
+    # characters and still be rejected, and memcached's CLIENT_ERROR for it
+    # must never reach a pipelined get.
+    def wire_length(key)
+      return key.bytesize unless Dalli::Protocol::Meta::KeyRegularizer.required?(key)
+
+      ((key.bytesize + 2) / 3) * 4
     end
 
     def prefix_length(digest)

data/lib/dalli/protocol/base.rb

@@ -51,6 +51,14 @@ module Dalli
         @raw_mode
       end
 
+      # True when values should come back as stored: the client is in raw mode,
+      # or this request passed raw: true. Retrieval paths use it to skip
+      # requesting flags, so a stored value is never deserialized or
+      # decompressed for a caller who asked for raw bytes.
+      def raw_request?(options)
+        raw_mode? || (options.is_a?(Hash) && options[:raw]) ? true : false
+      end
+
       # Chokepoint method for error handling and ensuring liveness
       def request(opkey, *args)
         verify_state(opkey)
@@ -142,7 +150,15 @@ module Dalli
         reconnect_on_pipeline_complete!
         values = nil
 
-        response_buffer.read
+        # Only the read is socket I/O. The caller's block runs below, outside
+        # this rescue, so an Errno or Timeout::Error it raises reaches the
+        # caller instead of being mistaken for a network failure (which closed
+        # the connection and retried the whole get_multi, yielding keys twice).
+        begin
+          response_buffer.read
+        rescue SystemCallError, *TIMEOUT_ERRORS, *SSL_ERRORS, EOFError => e
+          @connection_manager.error_on_request!(e)
+        end
 
         status, cas, key, value = response_buffer.process_single_getk_response
         # status is not nil only if we have a full response to parse
@@ -168,8 +184,6 @@ module Dalli
         end
 
         values || {}
-      rescue SystemCallError, *TIMEOUT_ERRORS, *SSL_ERRORS, EOFError => e
-        @connection_manager.error_on_request!(e)
       end
       # rubocop:enable Metrics/AbcSize, Metrics/CyclomaticComplexity, Metrics/PerceivedComplexity
 
@@ -183,10 +197,13 @@ module Dalli
         @connection_manager.abort_request!
         return true unless connected?
 
-        # Closes the connection, which ensures that our connection
-        # is in a clean state for future requests
-        @connection_manager.error_on_request!('External timeout')
-      rescue NetworkError
+        # Closes the connection, which ensures that our connection is in a
+        # clean state for future requests. The get_multi ran out of time
+        # overall rather than hitting a socket error, so it doesn't count
+        # toward socket_max_failures: two slow get_multis in a row would
+        # otherwise mark a healthy server down.
+        Dalli.logger.warn { "#{name} didn't finish a get_multi within socket_timeout; closing the connection" }
+        @connection_manager.close
         true
       end
 
@@ -314,6 +331,7 @@ module Dalli
         # Use clear (not reset) to keep pipeline_complete? = true, which is
         # the expected state before pipeline_response_setup is called.
         response_buffer.clear
+        response_buffer.raw = raw_request?(options)
 
         # The terminating noop is sent by pipeline_response_setup
         write(quiet_get_requests(keys, options, return_cas: return_cas))
@@ -325,6 +343,7 @@ module Dalli
       def pipelined_get_interleaved(keys, chunk_size, results, options = nil)
         # Initialize the response buffer for draining during send phase
         response_buffer.ensure_ready
+        response_buffer.raw = raw_request?(options)
 
         keys.each_slice(chunk_size) do |chunk|
           # Build and write this chunk of requests

data/lib/dalli/protocol/connection_manager.rb

@@ -35,7 +35,10 @@ module Dalli
         @request_in_progress = false
         @sock = nil
         @pid = nil
+        @write_buffer = []
+        @write_buffer_bytes = 0
 
+        @fail_count = 0
         reset_down_info
       end
 
@@ -51,7 +54,8 @@ module Dalli
         Dalli.logger.debug { "Dalli::Server#connect #{name}" }
 
         @sock = memcached_socket
-        @sock.sync = false # Enable buffered I/O for better performance
+        # Writes are buffered in @write_buffer instead; see WRITE_BUFFER_FLUSH_BYTES
+        @sock.sync = true
         @pid = Process.pid
         @request_in_progress = false
       rescue SystemCallError, *TIMEOUT_ERRORS, EOFError, SocketError => e
@@ -83,6 +87,8 @@ module Dalli
       def down!
         close
         log_down_detected
+        # Once down_retry_delay passes, the server gets a full set of attempts
+        @fail_count = 0
 
         @error = $ERROR_INFO&.class&.name
         @msg ||= $ERROR_INFO&.message
@@ -112,7 +118,7 @@ module Dalli
         return unless @sock
 
         begin
-          @sock.close
+          close_socket
         rescue StandardError
           nil
         ensure
@@ -123,10 +129,18 @@ module Dalli
           # @request_in_progress == true.
           @sock = nil
           @pid = nil
+          discard_write_buffer
           abort_request!
         end
       end
 
+      # A forked child shares the parent's connection, so it closes only its
+      # own file descriptor. Closing the TLS socket itself would send
+      # close_notify and end the parent's TLS session.
+      def close_socket
+        fork_detected? ? @sock.to_io.close : @sock.close
+      end
+
       def connected?
         !@sock.nil?
       end
@@ -145,6 +159,11 @@ module Dalli
         raise '[Dalli] No request in progress. This may be a bug in Dalli.' unless @request_in_progress
 
         @request_in_progress = false
+        # A completed request is what proves the server healthy again, so the
+        # failure count resets here rather than on reconnect: a server that
+        # accepts connections but never answers would otherwise reset it on
+        # every retry and never reach socket_max_failures.
+        @fail_count = 0
       end
 
       def abort_request!
@@ -152,6 +171,7 @@ module Dalli
       end
 
       def read_line
+        flush_write_buffer
         data = @sock.gets("\r\n")
         error_on_request!('EOF in read_line') if data.nil?
         data
@@ -161,14 +181,29 @@ module Dalli
 
       # JRuby doesn't support IO#timeout=, so use custom readfull implementation
       # CRuby 3.3+ has IO#timeout= which makes IO#read work with timeouts
+      # memcached can't store an item larger than 1 GiB (its -I maximum), so a
+      # reply claiming more (or a negative size) is malformed or hostile.
+      # IO#read allocates the full count up front, so check before reading.
+      MAX_READ_BYTES = (1024 * 1024 * 1024) + 2 # plus the value's trailing "\r\n"
+
+      def check_read_size!(count)
+        return if count.between?(0, MAX_READ_BYTES)
+
+        raise Dalli::DalliError, "Reply size #{count} from #{name} is out of range"
+      end
+
       if RUBY_ENGINE == 'jruby'
         def read(count)
+          check_read_size!(count)
+          flush_write_buffer
           @sock.readfull(count)
         rescue SystemCallError, *TIMEOUT_ERRORS, *SSL_ERRORS, EOFError => e
           error_on_request!(e)
         end
       else
         def read(count)
+          check_read_size!(count)
+          flush_write_buffer
           read_bytes(count)
         rescue SystemCallError, *TIMEOUT_ERRORS, *SSL_ERRORS, EOFError => e
           error_on_request!(e)
@@ -179,22 +214,34 @@ module Dalli
       # at the call site.
       alias read_exact read
 
+      # Requests are buffered here rather than in the socket's own IO buffer
+      # (the socket is sync). Ruby flushes an IO's write buffer when any process
+      # holding it closes or finalizes the IO, so after a fork a child could
+      # send bytes the parent had buffered but not sent yet: a quiet write would
+      # run twice, or the parent's replies would shift onto the wrong requests.
+      # A forked child discards this buffer instead. Like IO's own buffer, it
+      # is sent once it grows past this size, or when a reply is read.
+      WRITE_BUFFER_FLUSH_BYTES = 64 * 1024
+
       def write(bytes)
-        @sock.write(bytes)
+        @write_buffer << bytes
+        @write_buffer_bytes += bytes.bytesize
+        flush_write_buffer if @write_buffer_bytes >= WRITE_BUFFER_FLUSH_BYTES
+        bytes.bytesize
       rescue SystemCallError, *TIMEOUT_ERRORS, *SSL_ERRORS, IOError => e
         error_on_request!(e)
       end
 
       def flush
-        @sock.flush
+        flush_write_buffer
       rescue SystemCallError, *TIMEOUT_ERRORS, *SSL_ERRORS, IOError => e
         error_on_request!(e)
       end
 
       def flushed_write(bytes)
-        written = @sock.write(bytes)
-        @sock.flush
-        written
+        @write_buffer << bytes
+        flush_write_buffer
+        bytes.bytesize
       rescue SystemCallError, *TIMEOUT_ERRORS, *SSL_ERRORS, IOError => e
         error_on_request!(e)
       end
@@ -202,6 +249,7 @@ module Dalli
       # Non-blocking read.  Here to support the operation
       # of the get_multi operation
       def read_available(...)
+        flush_write_buffer
         @sock.read_available(...)
       end
 
@@ -228,8 +276,9 @@ module Dalli
         raise Dalli::RetryableNetworkError, message
       end
 
+      # Called on connect. Deliberately leaves @fail_count alone; see
+      # finish_request!.
       def reset_down_info
-        @fail_count = 0
         @down_at = nil
         @last_down_at = nil
         @msg = nil
@@ -254,7 +303,8 @@ module Dalli
       def reconnect_on_fork
         message = 'Fork detected, re-connecting child process...'
         Dalli.logger.info { message }
-        # Close socket on a fork and reconnect immediately
+        # Drops anything the parent had buffered, closes the inherited socket
+        # without touching the parent's connection, and reconnects immediately
         close
         establish_connection
       end
@@ -284,6 +334,37 @@ module Dalli
 
       private
 
+      # One write per flush. A TLS socket sends each argument of
+      # write(*parts) as its own record and system call, so the parts are
+      # joined first.
+      def flush_write_buffer
+        return if @write_buffer.empty?
+
+        @sock.write(@write_buffer.size == 1 ? @write_buffer.first : joined_write_buffer)
+        discard_write_buffer
+      end
+
+      # Joined as bytes: requests can mix UTF-8 and binary strings, which
+      # String#<< refuses to combine when both hold non-ASCII bytes
+      if String.method_defined?(:append_as_bytes)
+        def joined_write_buffer
+          out = String.new(capacity: @write_buffer_bytes, encoding: Encoding::BINARY)
+          @write_buffer.each { |part| out.append_as_bytes(part) }
+          out
+        end
+      else
+        def joined_write_buffer
+          out = String.new(capacity: @write_buffer_bytes, encoding: Encoding::BINARY)
+          @write_buffer.each { |part| out << part.b }
+          out
+        end
+      end
+
+      def discard_write_buffer
+        @write_buffer.clear
+        @write_buffer_bytes = 0
+      end
+
       # Reads exactly `count` bytes. IO#read(count) on a blocking socket blocks
       # until it has `count` bytes, accumulating across TCP chunks internally,
       # and only hands back a shorter (or nil) buffer when the stream hits EOF.

data/lib/dalli/protocol/meta.rb

@@ -38,20 +38,20 @@ module Dalli
                 RequestFormatter.plain_meta_get(key, skip_flags)
               end
         flushed_write(req)
-        response_processor.meta_get_with_value(cache_nils: cache_nils?(options))
+        response_processor.meta_get_with_value(cache_nils: cache_nils?(options), raw: raw_request?(options))
       end
 
       def quiet_get_request(key, options = nil)
         # Skip bitflags in raw mode - saves 2 bytes per request and skips parsing
-        RequestFormatter.meta_get(key: key, return_cas: true, quiet: true, skip_flags: raw_mode?,
+        RequestFormatter.meta_get(key: key, return_cas: true, quiet: true, skip_flags: raw_request?(options),
                                   **routing_token_kwargs(options))
       end
 
       # Same requests as quiet_get_request for each key, built in one pass
       # and without the trailing noop.
       def quiet_get_requests(keys, options = nil, return_cas: true)
-        RequestFormatter.multi_meta_get(keys, skip_flags: raw_mode?, return_cas: return_cas, terminate: false,
-                                              **routing_token_kwargs(options))
+        RequestFormatter.multi_meta_get(keys, skip_flags: raw_request?(options), return_cas: return_cas,
+                                              terminate: false, **routing_token_kwargs(options))
       end
 
       def gat(key, ttl, options = nil)
@@ -59,7 +59,7 @@ module Dalli
         skip_flags = raw_mode? || (options && options[:raw])
         req = RequestFormatter.meta_get(key: key, ttl: ttl, skip_flags: skip_flags, **routing_token_kwargs(options))
         flushed_write(req)
-        response_processor.meta_get_with_value(cache_nils: cache_nils?(options))
+        response_processor.meta_get_with_value(cache_nils: cache_nils?(options), raw: raw_request?(options))
       end
 
       def touch(key, ttl)
@@ -72,9 +72,10 @@ module Dalli
       # TODO: This is confusing, as there's a cas command in memcached
       # and this isn't it.  Maybe rename?  Maybe eliminate?
       def cas(key, options = nil)
-        req = RequestFormatter.meta_get(key: key, value: true, return_cas: true, **routing_token_kwargs(options))
+        req = RequestFormatter.meta_get(key: key, value: true, return_cas: true, skip_flags: raw_request?(options),
+                                        **routing_token_kwargs(options))
         flushed_write(req)
-        response_processor.meta_get_with_value_and_cas
+        response_processor.meta_get_with_value_and_cas(raw: raw_request?(options))
       end
 
       # Comprehensive meta get with support for all metadata flags.
@@ -103,7 +104,7 @@ module Dalli
       #   - :last_access - seconds since last access (only if return_last_access: true)
       def meta_get(key, options = {})
         req = RequestFormatter.meta_get(
-          key: key, value: true, return_cas: true,
+          key: key, value: true, return_cas: true, skip_flags: raw_request?(options),
           vivify_ttl: options[:vivify_ttl], recache_ttl: options[:recache_ttl],
           return_hit_status: options[:return_hit_status],
           return_last_access: options[:return_last_access],
@@ -115,7 +116,7 @@ module Dalli
         response_processor.meta_get_with_metadata(
           cache_nils: cache_nils?(options), return_hit_status: options[:return_hit_status],
           return_last_access: options[:return_last_access],
-          return_ttl_remaining: options[:return_ttl_remaining]
+          return_ttl_remaining: options[:return_ttl_remaining], raw: raw_request?(options)
         )
       end
 
@@ -266,7 +267,7 @@ module Dalli
       # response parsing to minimize per-key overhead. Avoids the PipelinedGetter
       # machinery (IO.select, response buffering, server grouping).
       def read_multi_req(keys, options = nil)
-        is_raw = raw_mode?
+        is_raw = raw_request?(options)
         buffer = RequestFormatter.multi_meta_get(keys, skip_flags: is_raw, **routing_token_kwargs(options))
         flushed_write(buffer)
         buffer.clear
@@ -282,7 +283,7 @@ module Dalli
       # Shared by both the single-server fast path and PipelinedGetter's
       # per-server-group request, so this one change covers both routes.
       def read_multi_with_metadata_req(keys, options = nil)
-        is_raw = raw_mode?
+        is_raw = raw_request?(options)
         buffer = RequestFormatter.multi_meta_get(keys, skip_flags: is_raw, return_cas: true,
                                                        **routing_token_kwargs(options))
         flushed_write(buffer)
@@ -299,7 +300,9 @@ module Dalli
           next unless line.start_with?('VA ')
 
           tokens = line.chomp!(TERMINATOR).split
-          value = @connection_manager.read(tokens[1].to_i + TERMINATOR.bytesize)&.chomp!(TERMINATOR)
+          size = tokens[1].to_i
+          response_processor.check_value_size!(size)
+          value = @connection_manager.read(size + TERMINATOR.bytesize)&.chomp!(TERMINATOR)
           stale = response_processor.stale_from_tokens(tokens)
           cas = response_processor.cas_from_tokens(tokens)
           bitflags = is_raw ? 0 : response_processor.bitflags_from_tokens(tokens)
@@ -326,7 +329,9 @@ module Dalli
 
       def parse_multi_get_value(line, key_index, is_raw)
         tokens = line.chomp!(TERMINATOR).split
-        value = @connection_manager.read(tokens[1].to_i + TERMINATOR.bytesize)&.chomp!(TERMINATOR)
+        size = tokens[1].to_i
+        response_processor.check_value_size!(size)
+        value = @connection_manager.read(size + TERMINATOR.bytesize)&.chomp!(TERMINATOR)
         raw_key = tokens[key_index]
         return unless raw_key
 

data/lib/dalli/protocol/request_formatter.rb

@@ -200,11 +200,11 @@ module Dalli
         # specifications to a proxy or router inbetween a client and a
         # memcached daemon."
         #
-        # Empty / nil tokens are treated as no-ops. CRLF and null bytes are
-        # rejected with ArgumentError to prevent the token from being used as a
-        # wire-protocol injection vector (e.g. "foo\r\nflush_all\r\n" would
-        # otherwise be parsed as a second command by memcached or any
-        # intermediate proxy/LB).
+        # Empty / nil tokens are treated as no-ops. Whitespace and control
+        # characters are rejected with ArgumentError so a token can't be used
+        # for wire-protocol injection: CRLF would add a second command (e.g.
+        # "foo\r\nflush_all\r\n"), and a space would add meta flags to this one
+        # (e.g. "foo T1" changing an item's TTL on a read).
         def routing_tokens(p_token: nil, l_token: nil)
           # Only an empty *String* is a no-op. Checking respond_to?(:empty?)
           # instead would also swallow p_token: [] / {} before the type check
@@ -261,15 +261,18 @@ module Dalli
 
         private
 
-        # Disallowed bytes: CR, LF, NUL. Any of these embedded in a routing
-        # token would let the caller inject a second wire-protocol command.
-        ROUTING_TOKEN_FORBIDDEN = /[\r\n\0]/
+        # Disallowed bytes: whitespace and control characters. CR, LF or NUL
+        # would let a token inject a second command; a space would let it add
+        # meta flags (T, N, I, q, ...) to the request it rides on.
+        ROUTING_TOKEN_FORBIDDEN = /[\x00-\x20\x7F]/
         private_constant :ROUTING_TOKEN_FORBIDDEN
 
         def validate_routing_token!(name, value)
           return if value.nil?
           raise ArgumentError, "#{name} must be a String, got #{value.class}" unless value.is_a?(String)
-          raise ArgumentError, "#{name} must not contain CRLF or null bytes" if value.match?(ROUTING_TOKEN_FORBIDDEN)
+          return unless value.match?(ROUTING_TOKEN_FORBIDDEN)
+
+          raise ArgumentError, "#{name} must not contain whitespace or control characters"
         end
 
         # Numeric flag values are written straight into the command line, so a

data/lib/dalli/protocol/response_buffer.rb

@@ -15,6 +15,9 @@ module Dalli
     # when advancing through parsed responses.
     ##
     class ResponseBuffer
+      # Whether the current pipeline was requested with raw: true
+      attr_writer :raw
+
       def initialize(io_source, response_processor)
         @io_source = io_source
         @response_processor = response_processor
@@ -39,7 +42,7 @@ module Dalli
       # Attempts to process a single response from the buffer,
       # advancing the offset past the consumed bytes.
       def process_single_getk_response
-        response = @response_processor.getk_response_from_buffer(@buffer, @offset)
+        response = @response_processor.getk_response_from_buffer(@buffer, @offset, raw: @raw || false)
         @offset += response.pop
         response
       end

data/lib/dalli/protocol/response_processor.rb

@@ -24,6 +24,8 @@ module Dalli
         SERVER_ERROR = 'SERVER_ERROR'
 
         VA_PREFIX = 'VA '
+        # memcached can't store an item larger than 1 GiB (its -I maximum)
+        MAX_VALUE_BYTES = 1024 * 1024 * 1024
         FLAGS_TOKEN_PREFIX = ' f'
         BYTE_B = 'b'.ord
         BYTE_C = 'c'.ord
@@ -46,12 +48,15 @@ module Dalli
           @value_marshaller = value_marshaller
         end
 
-        def meta_get_with_value(cache_nils: false)
+        # raw: true (a request made with raw: true) returns the value as stored,
+        # ignoring any flags in the reply. The request didn't ask for flags, so
+        # a reply carrying them anyway must not get the value deserialized.
+        def meta_get_with_value(cache_nils: false, raw: false)
           line = read_line
           # A hit ("VA <size> f<flags>") is parsed in place rather than split
           # into tokens, which saves several allocations on the hottest path.
           if line&.start_with?(VA_PREFIX)
-            return @value_marshaller.retrieve(read_data(size_from_va_line(line)), bitflags_from_va_line(line))
+            return retrieve(read_data(size_from_va_line(line)), raw ? 0 : bitflags_from_va_line(line), raw)
           end
 
           tokens = line&.split || []
@@ -59,7 +64,7 @@ module Dalli
           when EN
             cache_nils ? ::Dalli::NOT_FOUND : nil
           when VA # only reached for an unusually formatted hit line
-            @value_marshaller.retrieve(read_data(tokens[1].to_i), bitflags_from_tokens(tokens))
+            retrieve(read_data(tokens[1].to_i), raw ? 0 : bitflags_from_tokens(tokens), raw)
           when HD
             true
           else
@@ -67,14 +72,14 @@ module Dalli
           end
         end
 
-        def meta_get_with_value_and_cas
+        def meta_get_with_value_and_cas(raw: false)
           tokens = error_on_unexpected!(T_VA_EN_HD)
           return [nil, 0] if tokens.first == EN
 
           cas = cas_from_tokens(tokens)
           return [nil, cas] unless tokens.first == VA
 
-          [@value_marshaller.retrieve(read_data(tokens[1].to_i), bitflags_from_tokens(tokens)), cas]
+          [retrieve(read_data(tokens[1].to_i), raw ? 0 : bitflags_from_tokens(tokens), raw), cas]
         end
 
         def meta_get_without_value
@@ -94,13 +99,13 @@ module Dalli
         # Used by meta_get for comprehensive metadata retrieval.
         # Supports thundering herd protection (N/R flags) and metadata flags (h/l/u).
         def meta_get_with_metadata(cache_nils: false, return_hit_status: false, return_last_access: false,
-                                   return_ttl_remaining: false)
+                                   return_ttl_remaining: false, raw: false)
           tokens = error_on_unexpected!(T_VA_EN_HD)
           result = build_metadata_result(tokens)
           result[:hit_before] = hit_status_from_tokens(tokens) if return_hit_status
           result[:last_access] = last_access_from_tokens(tokens) if return_last_access
           result[:ttl_remaining] = ttl_remaining_from_tokens(tokens) if return_ttl_remaining
-          result[:value] = parse_value_from_tokens(tokens, cache_nils)
+          result[:value] = parse_value_from_tokens(tokens, cache_nils, raw)
           result
         end
 
@@ -117,11 +122,11 @@ module Dalli
           }
         end
 
-        def parse_value_from_tokens(tokens, cache_nils)
+        def parse_value_from_tokens(tokens, cache_nils, raw = false) # rubocop:disable Style/OptionalBooleanParameter
           return cache_nils ? ::Dalli::NOT_FOUND : nil if tokens.first == EN
           return unless tokens.first == VA
 
-          @value_marshaller.retrieve(read_data(tokens[1].to_i), bitflags_from_tokens(tokens))
+          retrieve(read_data(tokens[1].to_i), raw ? 0 : bitflags_from_tokens(tokens), raw)
         end
 
         def meta_set_with_cas
@@ -212,7 +217,9 @@ module Dalli
         # The remaining three values in the array are the ResponseHeader,
         # key, and value.
         ##
-        def getk_response_from_buffer(buf, offset = 0)
+        # raw: true returns values as stored, ignoring flags in the replies;
+        # see meta_get_with_value.
+        def getk_response_from_buffer(buf, offset = 0, raw: false)
           # Find the header terminator starting from offset
           term_idx = buf.byteindex(TERMINATOR, offset)
           return [0] unless term_idx
@@ -234,16 +241,20 @@ module Dalli
             when BYTE_B then base64 ||= token == 'b'
             end
           end
-          body_len = flag_int(size, 's')
-
-          # We have a complete response that has no body.
-          # This is either the response to the terminating
-          # noop or, if the status is not MN, an intermediate
-          # error response that needs to be discarded. A hit
-          # on an empty value (VA 0) still has a body -- just
-          # its terminator -- so it's parsed below as a value.
-          return [true, header_len] if body_len.zero? && tokens.first != VA
-
+          # memcached also gives the size positionally ("VA <size> ..."); use it
+          # when there's no s flag rather than reading the value as replies
+          body_len = size ? flag_int(size, 's') : positional_size(tokens)
+
+          # A complete response with no body. Only the terminating noop's MN
+          # ends the pipeline. Any other bodyless reply (CLIENT_ERROR,
+          # SERVER_ERROR, ...) answers one key's request: report it with a
+          # false status so the caller skips it and keeps reading. Treating it
+          # as the end would leave the remaining replies on the connection for
+          # later commands to read as their own. A hit on an empty value (VA 0)
+          # still has a body -- just its terminator -- so it's parsed below.
+          return [tokens.first == MN, header_len] if body_len.zero? && tokens.first != VA
+
+          check_value_size!(body_len)
           resp_size = header_len + body_len + TERMINATOR.length
           # The header is in the buffer, but the body is not.  As we don't have
           # a complete response, don't advance the buffer
@@ -252,12 +263,21 @@ module Dalli
           # The full response is in our buffer, so parse it and return
           # the values
           body = buf.byteslice(offset + header_len, body_len)
-          value = @value_marshaller.retrieve(body, flag_int(bitflags, 'f'))
-          key = key ? key.delete_prefix!('k') : 0
+          value = retrieve(body, raw ? 0 : flag_int(bitflags, 'f'), raw)
+          # A hit with no key can't be matched to a request: skip it (false
+          # status) rather than yield it, or take it for the end of the pipeline
+          return [false, resp_size] unless key
+
+          key = key.delete_prefix!('k')
           key = KeyRegularizer.decode(key) if base64
           [tokens.first == VA, flag_int(cas, 'c'), key, value, resp_size]
         end
 
+        # Values from a raw request are returned as stored
+        def retrieve(value, bitflags, raw)
+          raw ? value : @value_marshaller.retrieve(value, bitflags)
+        end
+
         # Integer value of a flag token such as "f123", or 0 when absent.
         # Strips the prefix in place, like value_from_tokens.
         def flag_int(token, flag)
@@ -346,6 +366,19 @@ module Dalli
           end
         end
 
+        # A pipelined reply claiming an impossible size would otherwise have the
+        # buffer wait for (and accumulate) that many bytes
+        # The size given right after "VA", or 0 when it isn't a number
+        def positional_size(tokens)
+          tokens.first == VA && tokens[1]&.match?(/\A\d+\z/) ? tokens[1].to_i : 0
+        end
+
+        def check_value_size!(size)
+          return if size.between?(0, MAX_VALUE_BYTES)
+
+          raise Dalli::DalliError, "Reply value size #{size} is out of range"
+        end
+
         def read_line
           @io_source.read_line&.chomp!(TERMINATOR)
         end
@@ -356,6 +389,9 @@ module Dalli
         end
 
         def read_data(data_size)
+          # Checked before the terminator is added: -1 or -2 would otherwise
+          # read 1 or 0 bytes and leave the connection out of step
+          check_value_size!(data_size)
           @io_source.read(data_size + TERMINATOR.bytesize)&.chomp!(TERMINATOR)
         end
       end

data/lib/dalli/protocol/value_compressor.rb

@@ -15,11 +15,17 @@ module Dalli
         compress: true,
         compressor: ::Dalli::Compressor,
         # min byte size to attempt compression
-        compression_min_size: 4 * 1024 # 4K
+        compression_min_size: 4 * 1024, # 4K
+        # max size a stored value may decompress to (nil: no limit). Guards
+        # against a small compressed value expanding into gigabytes on read.
+        decompressed_max_bytes: 128 * 1024 * 1024 # 128 MiB
       }.freeze
 
       OPTIONS = DEFAULTS.keys.freeze
 
+      KEYWORD_PARAMETER_TYPES = %i[key keyreq].freeze
+      private_constant :KEYWORD_PARAMETER_TYPES
+
       def initialize(client_options)
         @compression_options =
           DEFAULTS.merge(client_options.slice(*OPTIONS))
@@ -34,8 +40,16 @@ module Dalli
       end
 
       def retrieve(value, bitflags)
-        compressed = bitflags.anybits?(Flags::COMPRESSED)
-        compressed ? compressor.decompress(value) : value
+        return value unless bitflags.anybits?(Flags::COMPRESSED)
+
+        max_bytes = @compression_options[:decompressed_max_bytes]
+        # Custom compressors that only define decompress(data) keep working,
+        # without the limit
+        if max_bytes && compressor_accepts_max_bytes?
+          compressor.decompress(value, max_bytes: max_bytes)
+        else
+          compressor.decompress(value)
+        end
 
       # TODO: We likely want to move this rescue into the Dalli::Compressor / Dalli::GzipCompressor
       # itself, since not all compressors necessarily use Zlib.  For now keep it here, so the behavior
@@ -44,6 +58,19 @@ module Dalli
         raise UnmarshalError, "Unable to uncompress value: #{$ERROR_INFO.message}"
       end
 
+      def compressor_accepts_max_bytes?
+        return @compressor_accepts_max_bytes if defined?(@compressor_accepts_max_bytes)
+
+        @compressor_accepts_max_bytes =
+          begin
+            compressor.method(:decompress).parameters.any? do |type, name|
+              name == :max_bytes && KEYWORD_PARAMETER_TYPES.include?(type)
+            end
+          rescue NameError # an object without a reflectable decompress method
+            false
+          end
+      end
+
       def compress_by_default?
         @compression_options[:compress]
       end

data/lib/dalli/protocol/value_serializer.rb

@@ -100,7 +100,7 @@ module Dalli
         Dalli.logger.warn 'SECURITY WARNING: Dalli is using Marshal for serialization. ' \
                           'Marshal can execute arbitrary code during deserialization. ' \
                           'If your memcached server could be compromised, consider using ' \
-                          'a safer serializer like JSON: Dalli::Client.new(servers, serializer: JSON)'
+                          'a safer serializer: Dalli::Client.new(servers, serializer: Dalli::JSONSerializer)'
         @@marshal_warning_logged = true # rubocop:disable Style/ClassVars
       end
     end

data/lib/dalli/version.rb

@@ -1,7 +1,7 @@
 # frozen_string_literal: true
 
 module Dalli
-  VERSION = '5.1.2'
+  VERSION = '5.1.4'
 
   MIN_SUPPORTED_MEMCACHED_VERSION = '1.6.27'
 end

data/lib/dalli.rb

@@ -62,6 +62,7 @@ require_relative 'dalli/version'
 require_relative 'dalli/instrumentation'
 
 require_relative 'dalli/flags'
+require_relative 'dalli/json_serializer'
 require_relative 'dalli/compressor'
 require_relative 'dalli/client'
 require_relative 'dalli/key_manager'

metadata

@@ -1,7 +1,7 @@
 --- !ruby/object:Gem::Specification
 name: dalli
 version: !ruby/object:Gem::Version
-  version: 5.1.2
+  version: 5.1.4
 platform: ruby
 authors:
 - Peter M. Goldstein
@@ -42,6 +42,7 @@ files:
 - lib/dalli/compressor.rb
 - lib/dalli/flags.rb
 - lib/dalli/instrumentation.rb
+- lib/dalli/json_serializer.rb
 - lib/dalli/key_manager.rb
 - lib/dalli/options.rb
 - lib/dalli/pipelined_deleter.rb