dalli 3.2.15 → 4.0.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 1282508d83c234919dfb2c8a05c08d7069186663c83d4e3aaf040e6f503a2e9f
-  data.tar.gz: cb10108ea1d22c25630b5cfa36b6e038515e3f053e2354e3921718050138535b
+  metadata.gz: 86e39b2869b7c916472e88e6cbbfb6aa1f6533aa35c9e1f1c704cffcbc680f8a
+  data.tar.gz: bf2e56610c6bae187d561ccf09105e6f5b4a29eed1308d089745ef22cf5b673b
 SHA512:
-  metadata.gz: 614a4eebdaa9cee1d9dc6afdfd45c96ff1ec7eda12f471a12c6c1eca7866af971d907a7c288aeb5a72734b3f4f72bfb685ba58337da2ac50146ed74a018c0d0a
-  data.tar.gz: 8f577fef557f593d9cb6327a54ec1b5d3df8e554a5935ba7a7faadf86259248a60f29204b3ad5a295024b676ebc481e925f81f330095a73e6649e11d182b1127
+  metadata.gz: e2faab814abb4b25a53d87048f7e4e81e49c609bd64725cb18607670110da3d5b8038544598c44575831b9038500f9d8220a56ec28c16b9517a21d442212d12b
+  data.tar.gz: 3e5a7c326c908d72d2d8ce2211b27f67acebbbf97c279c5faf3ba2914ec7409df7f259a9082d82903cc9ae6b6716a137994aba849321546b8d3d3a35f8f7f5af

data/CHANGELOG.md

@@ -1,108 +1,30 @@
 Dalli Changelog
 =====================
 
-Unreleased
-==========
-
-3.2.15
-==========
-
-- Keep a connection shared by several threads or fibers in step during a `quiet` block (GHSA-5xvx-r8rg-5qm2)
-  - A `quiet` (or `multi`) block is quiet only for the thread or fiber that runs it. memcached still replies to a quiet request that fails (for example an `add` of a key that exists), and another thread or fiber using the same client while the block was open read that reply as its own, so its later requests could return another key's value. A non-quiet request now reads any pending quiet replies first
-  - Affects every supported line, with both protocols; with the binary protocol the connection stayed out of step after the block ended. Fixed in 5.2.4, 5.1.6, 5.0.12, 4.3.9 and 3.2.15
-- With `protocol: :meta`, read values by their size instead of up to the first CRLF (GHSA-5xvx-r8rg-5qm2)
-  - `get`, `gat` and `cas` read a value as one line, so a value containing CR/LF was cut short and the rest was parsed as the next reply. A stored value containing a crafted reply was then returned for a later read of a different key, with flags that could cause it to be deserialized
-  - Affects 3.2.0 to 3.2.14 with `protocol: :meta`; fixed in 3.2.15. The binary protocol (3.2's default) and 4.0 and later aren't affected
-- A per-request `raw: true` write no longer compresses the value (backport of #1213)
-  - Values at or above the compression threshold (4 KB by default) were compressed and flagged even with `raw: true`. Raw reads ignore flags, so reading them back with `raw: true` returned the compressed bytes
-  - Raw values are now stored exactly as given. Callers that pass `raw: true` without `compress: false` store large values uncompressed; Rails' `MemCacheStore` passes `compress: false` and isn't affected
-- Handle failed TLS handshakes like other connection failures, and warn when TLS verification is off (backport of #1215)
-  - A failed or timed-out handshake left the TCP socket open until GC and raised `OpenSSL::SSL::SSLError` directly, so the server was never marked down. The socket is now closed, and the error goes through the usual retry and `socket_max_failures` handling, raising `Dalli::NetworkError` (or `Dalli::RingError` once no server is available)
-  - Certificate and hostname checks come only from the `ssl_context` you pass, and a bare `OpenSSL::SSL::SSLContext.new` checks neither. Dalli now logs a warning once per process when `verify_mode` doesn't include `VERIFY_PEER` or `verify_hostname` is false; see the new TLS section in the README
-- A reply line longer than 8 KiB without a CRLF, or one cut short by the connection closing, is now a network error that closes the connection. Before, the line had no length limit (backport of #1216)
-
-3.2.14
-==========
-
-- Don't rely on memcached's auto-create for keys that need base64 encoding (GHSA-r47x-6p6r-mvcp)
-  - memcached stores an item made by auto-create (`ma` with `N`/`J`, or `mg` with `N`) for a base64-encoded key without its binary-key flag, and later replies name the item by its raw key bytes. A key containing spaces or CR/LF could then make a multi-get read the replies after it as different ones
-  - `incr` and `decr` with an initial value now create such counters with an add instead. Return values are unchanged
-  - The new `verify_reply_keys` option makes `get_multi` ignore replies for keys they didn't request, which guards against items created by older Dalli versions or other clients until memcached is upgraded. It's off by default, since it costs a multi-get about 15%
-  - Affects `protocol: :meta` since 3.2.0; fixed in 5.2.3, 5.1.5, 5.0.11, 4.3.8 and 3.2.14. memcached 1.6.45 and earlier still name these items by their raw key bytes
-
-3.2.13
-==========
-
-- Complete the fix for per-request `raw: true` on reads (GHSA-wr87-m4jw-29x5)
-  - `cas` and `cas!` accepted `raw: true` but didn't pass it to the read, so the value was still deserialized. They now honor it, in both protocols
-  - Affects 3.2.12 and earlier; fixed in 5.2.2, 5.1.4, 5.0.10, 4.3.7 and 3.2.13
-- Complete the fork-safety fix (GHSA-w39f-xq2m-4g8x)
-  - Fork detection used a pid cache refreshed by a `Process._fork` hook. When another library's fork hook ran first in the child (connection_pool, loaded before Dalli as in Rails, closes its connections there), Dalli didn't know it was in a forked child and ended the parent's TLS session. It now checks `Process.pid`
-  - Fixed in 4.3.7 and 3.2.13
-
-- Fix a regression in 3.2.12: a `get_multi` that didn't finish within `socket_timeout` counted toward `socket_max_failures`, so two slow `get_multi` calls in a row marked a healthy server down. It now just closes the connection
-- Handle malformed pipelined replies from a broken or hostile server or proxy: a hit with no key, or with no `s` flag, could leave the connection out of step or raise an error
-- With `decompressed_max_bytes` in effect, data after the end of a compressed value's stream was returned as part of the value. It's now ignored, as without the limit
-- With a `digest_class` whose digests aren't short hex strings, shortening a long key could loop forever. It now raises `ArgumentError`
-
-- Run the Tests and RuboCop workflows on pushes to `main` and the `*-stable` branches only, so a pull request's branch isn't tested twice (backport of #1198)
-
-3.2.12
-==========
-
-- Fix pipelined `get_multi` returning one key's value for another after an error reply (GHSA-p6pm-ch9v-44vx)
-  - The pipelined reply parser took any reply without a body for the end of the batch. An error reply for one key (`CLIENT_ERROR`, `SERVER_ERROR`, or `EN` from a proxy) ended it early, and the replies still on the connection were read as the replies to later commands, so a `get` could return another key's value
-  - Keys are now measured in bytes as sent (after base64 encoding when the meta protocol needs it). A key under 250 characters but over 250 bytes, for example one with many non-ASCII characters, passed validation:
-    - with the meta protocol, memcached rejected it with the error reply above
-    - with the binary protocol, memcached dropped the connection, and `get` or `get_multi` with that key retried forever
-  - Such keys are now truncated like other long keys; keys that worked before are unchanged
-  - The reply parser affects `protocol: :meta` since 3.2.0; the key length affects both protocols; fixed in 5.2.1, 5.1.3, 5.0.9, 4.3.6 and 3.2.12
-- Stop retrying an unresponsive server forever (GHSA-4qp6-2jcr-596v)
-  - A server that accepted connections but never answered (or dropped the connection on a request) was retried forever, hanging the caller, because each successful reconnect reset the failure count. Requests now fail after `socket_max_failures` attempts and the server is marked down, as when it can't be reached at all; it gets a full set of attempts again after `down_retry_delay`
-  - Affects all versions; fixed in 5.2.1, 5.1.3, 5.0.9, 4.3.6 and 3.2.12
-- Limit decompressed and reply value sizes (GHSA-3553-vcg5-72jw)
-  - Values flagged as compressed were inflated without limit, so a small stored value could expand to gigabytes on read, whatever the client's `compress` or `serializer` settings. The new `decompressed_max_bytes` option (default 128 MiB; `nil` disables it) makes a read that would pass it raise `Dalli::UnmarshalError`. Custom compressors whose `decompress` takes only the data keep working, without the limit
-  - The value size in a reply (a meta `VA` size or a binary body length) was used to read or buffer that many bytes, so a malicious server could make the client allocate gigabytes. Sizes over 1 GiB, memcached's largest item, now raise `Dalli::DalliError` before reading
-  - Affects all versions; fixed in 5.2.1, 5.1.3, 5.0.9, 4.3.6 and 3.2.12
-- Honor per-request `raw: true` on reads, and add `Dalli::JSONSerializer` (GHSA-wr87-m4jw-29x5)
-  - `get`, `gat` and `fetch` ignored a per-request `raw: true` and deserialized the value according to its stored flags, so a caller who asked for raw bytes could still have `Marshal.load` run on data someone else wrote to memcached. They now return the stored bytes
-  - The `raw` part affects all versions
-  - `serializer: JSON` reads values with `JSON.load`, which on json gem versions before 3.0 creates an object of the class named in a stored `json_class` key when the json additions are loaded. The new `Dalli::JSONSerializer` reads with `JSON.parse` and only returns plain JSON types. The README now recommends it; `serializer: JSON` itself is unchanged
-  - The `serializer: JSON` part affects any version used with the json gem before 3.0
-- Keep a forked child from closing the parent's TLS session (GHSA-w39f-xq2m-4g8x)
-  - With TLS, a forked child that closed the client (or reconnected after detecting the fork) sent a TLS close on the connection the parent was still using, ending the parent's session. A forked child now closes only its own copy of the socket
-  - Affects all versions with TLS; fixed in 5.2.1, 5.1.3, 5.0.9, 4.3.6 and 3.2.12
-
-- Fix three flaky tests: the `KeyManager` namespace key-length test, the Rack session freshness test, and `MemcachedManager`'s start and stop handling (backport of #1141, #1183)
-- Fix flaky failover tests: move their ports out of Linux's ephemeral port range, and wait for memcached to accept connections after starting it (backport of #1184)
-
-3.2.11
-==========
-
-- Fix multi-server `get_multi` stopping at an empty value (#1170)
-  - The pipelined reply parser took a hit on a zero-length value (`VA 0`) for the terminating `MN`, so that server's remaining keys were silently missing from the result
-  - Its value terminator and any replies not yet parsed stayed on the connection and were read as the replies to later commands on it. Those commands could fail with `Dalli::DalliError: Response error`, or a single-key `get` could silently return another key's value
-  - An empty value is now returned as `''`, like a single-key `get` and single-server `get_multi`
-  - Affects only clients using `protocol: :meta`; the default binary protocol is not affected
-  - Thanks to Julian Richard Contreras for this contribution
-
-3.2.10
-==========
-
-- Fix `Rack::Session::Dalli` with connection_pool 3.x, which only accepts keyword arguments (backport of #1051)
-- Add `logger` as a runtime dependency, since it is no longer a default gem as of Ruby 4.0 (backport of #1010, olleolleolle)
-- CI: patch memcached 1.5.22 so it builds on ubuntu-22.04, and run tests there (backport of #1033)
-
-3.2.9
-==========
-
-Security:
-
-- Fix memcached command injection through the `incr`/`decr` default with the meta protocol (GHSA-6wmv-xq9m-fmp7)
-  - With `protocol: :meta`, the `default` argument of `incr`/`decr` was written into the command without conversion, so a String containing CRLF injected additional memcached commands (e.g. `set`, `flush_all`) on the connection. The default binary protocol is not affected
-  - `default` must now be an Integer, or a String of decimal digits; anything else raises `ArgumentError` before a request is sent
-  - As defense in depth, the meta `RequestFormatter` now converts every numeric flag it writes (`D`, `J`, `N`, `T`) to an Integer
-  - Thanks to oss-security-shop for the report
+4.0.0
+==========
+
+BREAKING CHANGES:
+
+- Require Ruby 3.1+ (dropped support for Ruby 2.6, 2.7, and 3.0)
+- Removed `Dalli::Server` deprecated alias - use `Dalli::Protocol::Binary` instead
+- Removed `:compression` option - use `:compress` instead
+- Removed `close_on_fork` method - use `reconnect_on_fork` instead
+
+Other changes:
+
+- Add security warning when using default Marshal serializer (silence with `silence_marshal_warning: true`)
+- Add defense-in-depth input validation for stats command arguments
+- Add `string_fastpath` option to skip serialization for simple strings (byroot)
+- Meta protocol set performance improvement (danmayer)
+- Fix connection_pool 3.0 compatibility for Rack session store
+- Fix session recovery after deletion (stengineering0)
+- Fix cannot read response data included terminator `\r\n` when use meta protocol (matsubara0507)
+- Support SERVER_ERROR response from Memcached as per the [memcached spec](https://github.com/memcached/memcached/blob/e43364402195c8e822bb8f88755a60ab8bbed62a/doc/protocol.txt#L172) (grcooper)
+- Update Socket timeout handling to use Socket#timeout= when available (nickamorim)
+- Serializer: reraise all .load errors as UnmarshalError (olleolleolle)
+- Reconnect gracefully when a fork is detected instead of crashing (PatrickTulskie)
+- Update CI to test against memcached 1.6.40
 
 3.2.8
 ==========

data/Gemfile

@@ -5,7 +5,9 @@ source 'https://rubygems.org'
 gemspec
 
 group :development, :test do
+  gem 'cgi'
   gem 'connection_pool'
+  gem 'debug' unless RUBY_PLATFORM == 'java'
   gem 'minitest', '~> 5'
   gem 'rack', '~> 2.0', '>= 2.2.0'
   gem 'rake', '~> 13.0'

data/README.md

@@ -25,41 +25,6 @@ The name is a variant of Salvador Dali for his famous painting [The Persistence
 * [Forum](https://github.com/petergoldstein/dalli/discussions/categories/q-a) - If you have questions about Dalli, please post them here.
 * [Client API](https://www.rubydoc.info/gems/dalli) - Ruby documentation for the `Dalli::Client` API
 
-## Security Note
-
-By default, Dalli uses Ruby's Marshal for serialization. Deserializing untrusted data with Marshal can lead to remote code execution. If you cache user-controlled data, or your memcached server could be written to by someone else, consider using a safer serializer:
-
-```ruby
-Dalli::Client.new('localhost:11211', serializer: Dalli::JSONSerializer)
-```
-
-`Dalli::JSONSerializer` reads values with `JSON.parse`, so it only ever returns plain hashes, arrays, strings, numbers, booleans and `nil`. Passing the `JSON` module itself (`serializer: JSON`) reads values with `JSON.load`, which can create objects of other classes from a stored `json_class` key and is not recommended for data an attacker might write.
-
-Per-request `raw: true` returns the stored bytes without deserializing or decompressing them, on every read method that accepts it (`get` and `fetch`).
-
-### Counters and verify_reply_keys
-
-memcached names an item made by auto-create for a base64-encoded key (one with whitespace, control or non-ASCII characters) by its raw key bytes in later replies (with `protocol: :meta`). Since 3.2.14, Dalli creates counters for such keys with an add rather than auto-create. Items created before then, or by other clients, can still be named that way until memcached releases a fix (1.6.45 and earlier behave this way). If keys passed to `incr` or `decr` with an initial value can contain untrusted input, set `verify_reply_keys: true` until then: multi-gets then ignore replies for keys they didn't request, at a cost of about 15% per multi-get.
-
-## TLS
-
-Pass an `OpenSSL::SSL::SSLContext` as `ssl_context:` to connect over TLS. Certificate and hostname verification come only from that context: Dalli adds no checks of its own. A bare `OpenSSL::SSL::SSLContext.new` verifies nothing, so it accepts any certificate from any server. Configure the context to verify both:
-
-```ruby
-ssl_context = OpenSSL::SSL::SSLContext.new
-ssl_context.set_params(
-  verify_mode: OpenSSL::SSL::VERIFY_PEER,
-  verify_hostname: true,
-  ca_file: '/path/to/ca.crt' # omit to use the system's trusted certificates
-)
-
-Dalli::Client.new('memcached.example.com:11211', ssl_context: ssl_context)
-```
-
-Dalli logs a warning once per process when the context's `verify_mode` is `VERIFY_NONE`, or when it is `VERIFY_PEER` but `verify_hostname` is false.
-
-A failed TLS handshake counts as a failed connection attempt, like a refused connection: once `socket_max_failures` attempts have failed, the server is marked down and requests fail over to the other servers.
-
 ## Development
 
 After checking out the repo, run `bin/setup` to install dependencies. You can run `bin/console` for an interactive prompt that will allow you to experiment.

data/lib/dalli/client.rb

@@ -45,11 +45,6 @@ module Dalli
     #                   useful for injecting a FIPS compliant hash object.
     # - :protocol - one of either :binary or :meta, defaulting to :binary.  This sets the protocol that Dalli uses
     #               to communicate with memcached.
-    # - :verify_reply_keys - defaults to false. When true, get_multi only accepts replies for the keys it requested;
-    #                        after a reply naming another key, the rest of that server's replies are treated as misses.
-    #                        With the meta protocol, memcached can name an item by its raw key bytes when the item was
-    #                        created by auto-create from a base64-encoded key, which Dalli no longer does. Costs a
-    #                        multi-get about 15%.
     #
     def initialize(servers = nil, options = {})
       @normalized_servers = ::Dalli::ServersArgNormalizer.normalize_servers(servers)
@@ -160,8 +155,8 @@ module Dalli
     # - nil if the key did not exist.
     # - false if the value was changed by someone else.
     # - true if the value was successfully updated.
-    def cas(key, ttl = nil, req_options = nil, &block)
-      cas_core(key, false, ttl, req_options, &block)
+    def cas(key, ttl = nil, req_options = nil, &)
+      cas_core(key, false, ttl, req_options, &)
     end
 
     ##
@@ -171,8 +166,8 @@ module Dalli
     # Returns:
     # - false if the value was changed by someone else.
     # - true if the value was successfully updated.
-    def cas!(key, ttl = nil, req_options = nil, &block)
-      cas_core(key, true, ttl, req_options, &block)
+    def cas!(key, ttl = nil, req_options = nil, &)
+      cas_core(key, true, ttl, req_options, &)
     end
 
     ##
@@ -192,11 +187,6 @@ module Dalli
     # Any error replies will be discarded at the end of the block, and
     # Dalli client methods invoked inside the block will not
     # have return values
-    #
-    # The block is quiet only for the thread or fiber that runs it. Another
-    # thread or fiber sharing this client can make requests meanwhile; a
-    # non-quiet one first reads the block's pending replies with a noop and
-    # discards them, so it reads only its own.
     def quiet
       old = Thread.current[::Dalli::QUIET]
       Thread.current[::Dalli::QUIET] = true
@@ -267,8 +257,6 @@ module Dalli
     ##
     # Incr adds the given amount to the counter on the memcached server.
     # Amt must be a positive integer value.
-    # Default, if given, must be an Integer (or a String of decimal digits);
-    # anything else raises ArgumentError.
     #
     # If default is nil, the counter must already exist or the operation
     # will fail and will return nil.  Otherwise this method will return
@@ -281,7 +269,6 @@ module Dalli
     # If the value already exists, it must have been set with raw: true
     def incr(key, amt = 1, ttl = nil, default = nil)
       check_positive!(amt)
-      validate_integer!(:default, default)
 
       perform(:incr, key, amt.to_i, ttl_or_default(ttl), default)
     end
@@ -289,8 +276,6 @@ module Dalli
     ##
     # Decr subtracts the given amount from the counter on the memcached server.
     # Amt must be a positive integer value.
-    # Default, if given, must be an Integer (or a String of decimal digits);
-    # anything else raises ArgumentError.
     #
     # memcached counters are unsigned and cannot hold negative values.  Calling
     # decr on a counter which is 0 will just return 0.
@@ -306,7 +291,6 @@ module Dalli
     # If the value already exists, it must have been set with raw: true
     def decr(key, amt = 1, ttl = nil, default = nil)
       check_positive!(amt)
-      validate_integer!(:default, default)
 
       perform(:decr, key, amt.to_i, ttl_or_default(ttl), default)
     end
@@ -389,17 +373,8 @@ module Dalli
       raise ArgumentError, "Positive values only: #{amt}" if amt.negative?
     end
 
-    # Numeric arguments that become meta protocol flags (GHSA-6wmv-xq9m-fmp7).
-    # Checked before the request starts so a bad value raises a clean
-    # ArgumentError, instead of Protocol::Base#request logging it as
-    # unexpected and closing the connection.
-    def validate_integer!(name, value)
-      Dalli::Protocol::Meta::RequestFormatter.integer_flag(name, value) unless value.nil?
-    end
-
     def cas_core(key, always_set, ttl = nil, req_options = nil)
-      # req_options reaches the read too, so raw: true returns the stored bytes
-      (value, cas) = perform(:cas, key, req_options)
+      (value, cas) = perform(:cas, key)
       return if value.nil? && !always_set
 
       newvalue = yield(value)

data/lib/dalli/compressor.rb

@@ -13,54 +13,8 @@ module Dalli
       Zlib::Deflate.deflate(data)
     end
 
-    # DEFLATE can't expand data by more than about 1032:1
-    MAX_EXPANSION = 1032
-    # Compressed input is fed to zlib this many bytes at a time. MRI's zlib
-    # yields the output in small chunks; one that returns it instead still
-    # produces at most MAX_EXPANSION times this per step.
-    INFLATE_SLICE_BYTES = 64 * 1024
-
-    # max_bytes caps the decompressed size; inflating stops and raises as soon
-    # as the output passes it, so a small compressed value can't expand into
-    # gigabytes of memory.
-    def self.decompress(data, max_bytes: nil)
-      return Zlib::Inflate.inflate(data) unless may_exceed?(data, max_bytes)
-
-      inflate_within_limit(Zlib::Inflate.new, data, max_bytes)
-    end
-
-    # False when even the largest possible expansion stays within max_bytes,
-    # so the value can be inflated in one step
-    def self.may_exceed?(data, max_bytes)
-      max_bytes ? data.bytesize * MAX_EXPANSION > max_bytes : false
-    end
-
-    def self.inflate_within_limit(inflater, data, max_bytes)
-      out = String.new(encoding: Encoding::BINARY)
-      0.step(data.bytesize - 1, INFLATE_SLICE_BYTES) do |pos|
-        inflate_slice(inflater, data.byteslice(pos, INFLATE_SLICE_BYTES), out, max_bytes)
-        # Anything after the end of the stream isn't part of the value, as
-        # with Zlib::Inflate.inflate; finish would return it as output
-        break if inflater.finished?
-      end
-      return out if inflater.finished?
-
-      append_within_limit(out, inflater.finish, max_bytes)
-    ensure
-      inflater.close
-    end
-
-    # Both yielded and returned output count toward the limit
-    def self.inflate_slice(inflater, slice, out, max_bytes)
-      rest = inflater.inflate(slice) { |chunk| append_within_limit(out, chunk, max_bytes) }
-      append_within_limit(out, rest, max_bytes)
-    end
-
-    def self.append_within_limit(out, chunk, max_bytes)
-      out << chunk if chunk
-      return out if out.bytesize <= max_bytes
-
-      raise Dalli::UnmarshalError, "Decompressed value exceeds #{max_bytes} bytes (decompressed_max_bytes)"
+    def self.decompress(data)
+      Zlib::Inflate.inflate(data)
     end
   end
 
@@ -78,11 +32,9 @@ module Dalli
       io.string
     end
 
-    def self.decompress(data, max_bytes: nil)
-      return Zlib::GzipReader.new(StringIO.new(data, 'rb')).read unless Compressor.may_exceed?(data, max_bytes)
-
-      # MAX_WBITS + 16 makes zlib read the gzip format (and check its CRC)
-      Compressor.inflate_within_limit(Zlib::Inflate.new(Zlib::MAX_WBITS + 16), data, max_bytes)
+    def self.decompress(data)
+      io = StringIO.new(data, 'rb')
+      Zlib::GzipReader.new(io).read
     end
   end
 end

data/lib/dalli/key_manager.rb

@@ -30,13 +30,10 @@ module Dalli
 
     def initialize(client_options)
       @key_options =
-        DEFAULTS.merge(client_options.select { |k, _| OPTIONS.include?(k) })
+        DEFAULTS.merge(client_options.slice(*OPTIONS))
       validate_digest_class_option(@key_options)
 
       @namespace = namespace_from_options
-      # The meta protocol base64-encodes keys with whitespace or non-ASCII
-      # characters, so their length on the wire differs from their length here
-      @meta_protocol = client_options[:protocol]&.to_s == 'meta'
     end
 
     ##
@@ -54,7 +51,7 @@ module Dalli
       raise ArgumentError, 'key cannot be blank' unless key&.length&.positive?
 
       key = key_with_namespace(key)
-      wire_length(key) > MAX_KEY_LENGTH ? truncated_key(key) : key
+      key.length > MAX_KEY_LENGTH ? truncated_key(key) : key
     end
 
     ##
@@ -80,7 +77,7 @@ module Dalli
     def namespace_regexp
       return /\A#{Regexp.escape(evaluate_namespace)}:/ if namespace.is_a?(Proc)
 
-      @namespace_regexp ||= /\A#{Regexp.escape(namespace)}:/.freeze unless namespace.nil?
+      @namespace_regexp ||= /\A#{Regexp.escape(namespace)}:/ unless namespace.nil?
     end
 
     def validate_digest_class_option(opts)
@@ -110,30 +107,7 @@ module Dalli
     ##
     def truncated_key(key)
       digest = digest_class.hexdigest(key)
-      prefix = key[0, [prefix_length(digest), 0].max]
-      truncated = "#{prefix}#{TRUNCATED_KEY_SEPARATOR}#{digest}"
-      # A prefix with whitespace or non-ASCII characters sends the whole key
-      # base64-encoded, which can still be too long on the wire; shorten it.
-      while wire_length(truncated) > MAX_KEY_LENGTH && !prefix.empty?
-        prefix = prefix[0...-1]
-        truncated = "#{prefix}#{TRUNCATED_KEY_SEPARATOR}#{digest}"
-      end
-      return truncated if wire_length(truncated) <= MAX_KEY_LENGTH
-
-      # Only possible with a digest_class whose digests aren't short hex strings
-      raise ArgumentError, "#{digest_class} digests are too long to fit in a truncated key"
-    end
-
-    # memcached's key limit applies to the key as sent: in bytes, and, with
-    # the meta protocol, after base64 encoding when the key needs it. A key
-    # within the limit in characters can still be too long. With the meta
-    # protocol memcached rejects it with a CLIENT_ERROR that must never reach
-    # a pipelined get; with the binary protocol it drops the connection.
-    def wire_length(key)
-      return key.bytesize unless @meta_protocol
-
-      encoded, = Dalli::Protocol::Meta::KeyRegularizer.encode(key)
-      encoded.bytesize
+      "#{key[0, prefix_length(digest)]}#{TRUNCATED_KEY_SEPARATOR}#{digest}"
     end
 
     def prefix_length(digest)

data/lib/dalli/protocol/base.rb

@@ -1,7 +1,6 @@
 # frozen_string_literal: true
 
 require 'forwardable'
-require 'set'
 require 'socket'
 require 'timeout'
 
@@ -24,28 +23,17 @@ module Dalli
       def initialize(attribs, client_options = {})
         hostname, port, socket_type, @weight, user_creds = ServerConfigParser.parse(attribs)
         @options = client_options.merge(user_creds)
-        @verify_reply_keys = client_options[:verify_reply_keys] ? true : false
         @value_marshaller = ValueMarshaller.new(@options)
         @connection_manager = ConnectionManager.new(hostname, port, socket_type, @options)
       end
 
-      # True when this request passed raw: true. A stored value is then
-      # returned as stored, never deserialized or decompressed.
-      def raw_request?(options)
-        options.is_a?(Hash) && options[:raw] ? true : false
-      end
-
       # Chokepoint method for error handling and ensuring liveness
       def request(opkey, *args)
         verify_state(opkey)
 
         begin
           @connection_manager.start_request!
-          drain_pending_quiet_responses(opkey)
           response = send(opkey, *args)
-          # Quiet requests can leave replies on the socket; remember that so a
-          # later non-quiet request knows to read them first
-          @connection_manager.quiet_responses_pending = true if quiet? && opkey != :noop
 
           # pipelined_get emit query but doesn't read the response(s)
           @connection_manager.finish_request! unless opkey == :pipelined_get
@@ -111,7 +99,7 @@ module Dalli
 
           # If the status is ok and the key is not nil, then this is a
           # getkq response with a value that we want to set in the response hash
-          break unless keep_reading?(values, key, value, cas)
+          values[key] = [value, cas] unless key.nil?
 
           # Get the next response from the buffer
           status, cas, key, value = response_buffer.process_single_getk_response
@@ -130,16 +118,13 @@ module Dalli
       def pipeline_abort
         response_buffer.clear
         @connection_manager.abort_request!
-        return unless connected?
-
-        # Closes the connection, which ensures that our connection is in a
-        # clean state for future requests. The get_multi ran out of time
-        # overall rather than hitting a socket error, so it doesn't count
-        # toward socket_max_failures: two slow get_multis in a row would
-        # otherwise mark a healthy server down.
-        Dalli.logger.warn { "#{name} didn't finish a get_multi within socket_timeout; closing the connection" }
-        @connection_manager.close
-        nil
+        return true unless connected?
+
+        # Closes the connection, which ensures that our connection
+        # is in a clean state for future requests
+        @connection_manager.error_on_request!('External timeout')
+      rescue NetworkError
+        true
       end
 
       # Did the last call to #pipeline_response_setup complete successfully?
@@ -169,17 +154,7 @@ module Dalli
       private
 
       ALLOWED_QUIET_OPS = %i[add replace set delete incr decr append prepend flush noop].freeze
-
-      # Reads the replies still pending from quiet requests before any
-      # non-quiet request, which reads a reply of its own. A quiet block reads
-      # them at its end, but the connection is shared: another thread or fiber
-      # (quiet? is fiber-local) can make a request while the block is still
-      # open, and would otherwise read the block's replies as its own.
-      def drain_pending_quiet_responses(opkey)
-        return unless @connection_manager.quiet_responses_pending? && opkey != :noop && !quiet?
-
-        noop
-      end
+      private_constant :ALLOWED_QUIET_OPS
 
       def verify_allowed_quiet!(opkey)
         return if ALLOWED_QUIET_OPS.include?(opkey)
@@ -238,7 +213,6 @@ module Dalli
       end
 
       def pipelined_get(keys)
-        expect_reply_keys(keys)
         req = +''
         keys.each do |key|
           req << quiet_get_request(key)
@@ -251,50 +225,6 @@ module Dalli
         @response_buffer ||= ResponseBuffer.new(@connection_manager, response_processor)
       end
 
-      # A multi-get only accepts replies for the keys it asked for. With the
-      # meta protocol, memcached names an item created by auto-create (ma or mg
-      # with N) for a base64-encoded key by its raw bytes, without the b flag;
-      # such a key can contain spaces or CR/LF, which split the reply header and
-      # make the replies after it parse as different ones. A key that wasn't
-      # requested is how that shows up. Only with the verify_reply_keys option,
-      # since building the set costs a multi-get about 15%.
-      def expect_reply_keys(keys)
-        @expected_reply_keys = (keys.to_set { |k| reply_key_form(k) } if @verify_reply_keys)
-      end
-
-      # Stores a getkq reply's value and returns true, unless its key wasn't
-      # requested: then the replies are out of step, so what was read so far is
-      # kept, the rest dropped, and false returned to stop reading
-      def keep_reading?(values, key, value, cas)
-        return true if key.nil?
-
-        if unexpected_reply_key?(key)
-          abandon_pipeline
-          return false
-        end
-
-        values[key] = [value, cas]
-        true
-      end
-
-      def unexpected_reply_key?(key)
-        !key.nil? && @expected_reply_keys && !@expected_reply_keys.include?(reply_key_form(key))
-      end
-
-      # Keys compared as bytes; ASCII keys compare equal in any encoding as is
-      def reply_key_form(key)
-        key.ascii_only? ? key : key.b
-      end
-
-      # Drops the rest of this server's replies and closes the connection so
-      # nothing unread is left on it. Not a socket failure, so it doesn't count
-      # toward socket_max_failures; the keys not read are simply missing.
-      def abandon_pipeline
-        Dalli.logger.warn { "#{name} replied to a get_multi with a key that wasn't requested; closing the connection" }
-        response_buffer.clear
-        @connection_manager.close
-      end
-
       # Called after the noop response is received at the end of a set
       # of pipelined gets
       def finish_pipeline

data/lib/dalli/protocol/binary/response_processor.rb

@@ -66,9 +66,7 @@ module Dalli
           raise Dalli::DalliError, "Response error #{resp_header.status}: #{RESPONSE_CODES[resp_header.status]}"
         end
 
-        # raw: true returns the value as stored, without deserializing or
-        # decompressing it, whatever flags it was stored with
-        def get(cache_nils: false, raw: false)
+        def get(cache_nils: false)
           resp_header, body = read_response
 
           return false if resp_header.not_stored? # Not stored, normal status for add operation
@@ -77,7 +75,7 @@ module Dalli
           raise_on_not_ok!(resp_header)
           return true unless body
 
-          unpack_response_body(resp_header, body, !raw).last
+          unpack_response_body(resp_header, body, true).last
         end
 
         ##
@@ -102,8 +100,7 @@ module Dalli
           true
         end
 
-        # raw: true returns the value as stored; see #get
-        def data_cas_response(raw: false)
+        def data_cas_response
           resp_header, body = read_response
           return [nil, resp_header.cas] if resp_header.not_found?
           return [nil, false] if resp_header.not_stored?
@@ -111,7 +108,7 @@ module Dalli
           raise_on_not_ok!(resp_header)
           return [nil, resp_header.cas] unless body
 
-          [unpack_response_body(resp_header, body, !raw).last, resp_header.cas]
+          [unpack_response_body(resp_header, body, true).last, resp_header.cas]
         end
 
         # Returns the new value for the key, if found and updated
@@ -212,17 +209,6 @@ module Dalli
         # The remaining three values in the array are the ResponseHeader,
         # key, and value.
         ##
-        # memcached can't store an item larger than 1 GiB (its -I maximum). A
-        # pipelined reply claiming an impossible size would otherwise have the
-        # buffer wait for (and accumulate) that many bytes.
-        MAX_BODY_BYTES = 1024 * 1024 * 1024
-
-        def check_body_size!(size)
-          return if size <= MAX_BODY_BYTES
-
-          raise Dalli::DalliError, "Reply body size #{size} is out of range"
-        end
-
         def getk_response_from_buffer(buf)
           # There's no header in the buffer, so don't advance
           return [0, nil, nil, nil, nil] unless contains_header?(buf)
@@ -236,8 +222,6 @@ module Dalli
           # error response that needs to be discarded.
           return [ResponseHeader::SIZE, resp_header.ok?, resp_header.cas, nil, nil] if body_len.zero?
 
-          check_body_size!(body_len)
-
           resp_size = ResponseHeader::SIZE + body_len
           # The header is in the buffer, but the body is not.  As we don't have
           # a complete response, don't advance the buffer