dalli 5.1.2 → 5.2.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 53113b610a737a8a40a4f830c24251218161e36a83c01e3cf92b85e74ce2bbf1
-  data.tar.gz: 510c940cb0ee92a95e77939ffe277fc1a299539dff69d7e70cb2a8ca621ae94f
+  metadata.gz: 23a3890b1ca95b3668eea5b0f2f6eabfb3ad6062daebf5723e5b99c2c7a74854
+  data.tar.gz: 4f0c1775badbf133dab91d32a19f84ffafab7f1c0a6c9d7c195f968bed01c089
 SHA512:
-  metadata.gz: eacfdd099e2236d46abc8c445467f34088a23455cf42fa9cef9a328dfc64b474c0456063c2b40fb7e958ae96a455c63bf29ea8ad118501f64a46090364999d68
-  data.tar.gz: c0327598c3d0384e50bc132ef44cd5ccb94b346969eeee4b2235d43c7e31f2cde4fc9867b81c8f5ceffb6c624a01a1b3ea8de44a6ac856521fbcaa714d2a516d
+  metadata.gz: b1624d8d1024956e90ab86a5d5b7f56833d9d195b95d2e8f7ed92774775eebb5ae19baf798773e17ce8fdb7b8fa5abf728a5899d085ecd11335fe11daac52bdb
+  data.tar.gz: 76c4e96a6ac88ec7ce86d213d58bf4f709f6ebc52e747ffa05661343173c654696e6e2625090b14d31874f160f3778a750e9cd524cb4f8985ea86716f267793e

data/CHANGELOG.md

@@ -4,9 +4,47 @@ Dalli Changelog
 Unreleased
 ==========
 
-5.1.2
+5.2.0
 ==========
 
+Features:
+
+- Add `Dalli::CacheResult`, returned by the new `#get_with_metadata_result` and `#get_multi_with_metadata_result` (#1156)
+  - An opt-in, typed view over the Hash that `#get_with_metadata` and `#get_multi_with_metadata` already return; those methods are unchanged, and changing their return type is left for a future major version (#1151)
+  - `value`, `cas`, `hit_before`, `last_access` and `ttl_remaining` readers (the last three are `nil` unless requested), plus `miss?`, `hit?`, `stale?`, `won_recache?` and `lost_recache?` predicates
+  - Results are frozen, and a Hash that claims to be both a miss and stale raises `ArgumentError`
+  - `#get_multi_with_metadata_result` omits misses and accepts `req_options:`, like `#get_multi_with_metadata`; it has no block form
+- Add the opt-in `defer_drain` client option (#1168)
+  - With `defer_drain: true`, a `quiet`/`multi` block no longer waits at its end for the replies to its requests. The requests are still sent right away; the replies are read with one noop per server just before the next non-quiet request to that server
+  - Adds `Client#drain_deferred_responses`, to drain at a boundary of the caller's choosing (the end of a web request or job)
+  - Tradeoff: an error reply to a quiet request surfaces (and is discarded) at that later point, not at the end of the block
+  - Off by default, so existing behavior is unchanged
+  - Replies memcached sends for quiet requests wait on the connection until they're drained, so a client that only makes quiet requests should call `drain_deferred_responses` periodically; see "Deferred Draining" in the README
+  - Thanks to Julian Richard Contreras and Matt Dick for this contribution
+
+Performance:
+
+- Skip the empty read at the end of each pipelined `read_available` call (#1164)
+  - A read shorter than the chunk size means the socket is drained, so the extra `read_nonblock` that only returned `:wait_readable` is gone; the `IO.select` loop still picks up anything that arrives later
+  - Saves one syscall per server per multi-server `get_multi`; on 4 servers a 3-key `get_multi` goes from 16% to 5% slower than 2.7.11 in the #930 benchmark
+  - TLS sockets also check `SSLSocket#pending`, since `IO.select` cannot see data OpenSSL has buffered
+  - Thanks to Julian Richard Contreras for this contribution
+- Reduce Ruby overhead on plain `set` and `delete` (#1166)
+  - A `set` with no CAS, routing tokens or quiet mode builds its `ms` request with one string interpolation (`RequestFormatter.plain_meta_set`), and a `delete` with no options uses `plain_meta_delete`; both produce the same bytes as before
+  - `HD c<cas>`, `HD` and `NF` replies are parsed without splitting them into tokens
+  - Allocations per `set` drop from 22 to 18, and per `delete` from 12 to 7
+  - Thanks to Julian Richard Contreras for this contribution
+- Only drain servers that were sent quiet requests when a `quiet`/`multi` block ends (#1167)
+  - Ending the block used to send a noop to every server in the ring and wait for each reply, one after another, so its cost grew with ring size even when the block touched one server (or none)
+  - Each connection now records when a quiet request is written, and the block's drain skips the others; servers that were never connected are no longer connected just to be drained
+  - With ~300us of network round trip, `multi { set }` on a 16-server ring goes from 4.9ms to 0.32ms; an empty block no longer touches the network
+  - Thanks to Julian Richard Contreras for this contribution
+- Reduce Ruby overhead in `get_multi` reply parsing and key routing (#1169)
+  - `VA` reply headers are read in place instead of being split into tokens, on both the multi-server (pipelined) and single-server paths; results are unchanged, and unusual headers still take the token path
+  - A key's server is found through a bucket table over the ring's continuum instead of a binary search, and the key's own server is tried before the failover loop; the server chosen for every key is unchanged
+  - Allocations for a 100-key `get_multi` on 4 servers drop from 1,500 to 1,104; over loopback it goes from 301us to 243us
+  - Thanks to Julian Richard Contreras for this contribution
+
 Bug fixes:
 
 - Fix multi-server `get_multi` stopping at an empty value (#1170)

data/README.md

@@ -49,6 +49,32 @@ Dalli::Client.new('localhost:11211', namespace: 'myapp', namespace_separator: '/
 
 The separator must be a single non-alphanumeric character. Valid examples: `:`, `/`, `|`, `.`, `-`, `_`, `#`
 
+### Deferred Draining
+
+By default, a `quiet` (or `multi`) block ends by waiting for the replies memcached still sends for its quiet requests, one round trip for each server the block wrote to. With `defer_drain: true`, the block returns without waiting:
+
+```ruby
+dc = Dalli::Client.new('localhost:11211', defer_drain: true)
+
+dc.quiet { dc.set('key', 'value') } # returns without a round trip
+dc.get('key')                       # first drains that connection with a noop, then gets the value
+```
+
+The requests are still sent right away. Their replies stay unread on the connection until the next non-quiet request to that server, which reads them first with one `noop`. Error replies to quiet requests are discarded either way. With `defer_drain` they're discarded at that later point instead of at the end of the block.
+
+**Replies accumulate until a non-quiet request drains them.** memcached still replies to some quiet requests: `NF` for a quiet `delete`, `incr` or `decr` of a missing key, `NS` for an `add` or `replace` that didn't store, and error replies. Those replies wait on the socket. A client that mixes quiet writes with reads (or other non-quiet requests) to the same servers drains them as it goes, and needs nothing more.
+
+A client that only ever makes quiet requests, such as a worker that only issues quiet deletes, never sends that next request. Its unread replies keep growing, and if they fill the connection's socket buffers, the connection can stall until a socket timeout closes it and Dalli reconnects. Workloads like that should call `drain_deferred_responses` periodically, for example after each batch or job:
+
+```ruby
+jobs.each do |job|
+  job.keys.each { |key| dc.quiet { dc.delete(key) } }
+  dc.drain_deferred_responses # reads and discards the replies waiting on each server
+end
+```
+
+`drain_deferred_responses` only sends a `noop` to servers that were sent quiet requests since they were last drained, so calling it when nothing is pending costs nothing.
+
 ## Security Note
 
 By default, Dalli uses Ruby's Marshal for serialization. Deserializing untrusted data with Marshal can lead to remote code execution. If you cache user-controlled data, consider using a safer serializer:

data/lib/dalli/cache_result.rb

@@ -0,0 +1,65 @@
+# frozen_string_literal: true
+
+module Dalli
+  ##
+  # An immutable, typed view over the Hash returned by Client#get_with_metadata
+  # and Client#get_multi_with_metadata.
+  #
+  # This is an interim adapter (see #1130 / #1151): #get_with_metadata and
+  # #get_multi_with_metadata keep returning a Hash for now, since changing
+  # their return type is a breaking change reserved for a future major
+  # version. #get_with_metadata_result and #get_multi_with_metadata_result
+  # wrap that Hash in this type instead, so the structured shape can be
+  # validated by callers ahead of that change.
+  ##
+  class CacheResult
+    attr_reader :value, :cas, :hit_before, :last_access, :ttl_remaining
+
+    # @param hash [Hash] a Hash as returned by Client#get_with_metadata, or a
+    #   single value from the Hash returned by Client#get_multi_with_metadata
+    def initialize(hash)
+      @value = hash[:value]
+      @cas = hash[:cas]
+      @miss = hash.fetch(:miss, false)
+      @stale = hash.fetch(:stale, false)
+      @won_recache = hash.fetch(:won_recache, false)
+      @lost_recache = hash.fetch(:lost_recache, false)
+      @hit_before = hash[:hit_before]
+      @last_access = hash[:last_access]
+      @ttl_remaining = hash[:ttl_remaining]
+
+      raise ArgumentError, 'a result cannot be both a miss and stale' if @miss && @stale
+
+      freeze
+    end
+
+    # True when the key did not exist. Prefer this over a nil #value, which
+    # cannot distinguish a miss from a stored nil under cache_nils.
+    def miss?
+      @miss
+    end
+
+    # The inverse of #miss?.
+    def hit?
+      !@miss
+    end
+
+    # True when the item is a tombstone (marked stale via the meta protocol's
+    # invalidate flag) rather than removed. Not a miss: #value is still the
+    # last-known value unless the tombstone also dropped it.
+    def stale?
+      @stale
+    end
+
+    # True if this call won the right to regenerate the value under
+    # thundering-herd protection (see Client#fetch_with_lock).
+    def won_recache?
+      @won_recache
+    end
+
+    # True if another client already won the recache race for this key.
+    def lost_recache?
+      @lost_recache
+    end
+  end
+end

data/lib/dalli/client.rb

@@ -50,6 +50,14 @@ module Dalli
     #                        +:include+ logs the full operation and key(s), +:obfuscate+ replaces keys with "?",
     #                        +nil+ (default) omits the attribute entirely.
     # - :otel_peer_service - when set, adds a +peer.service+ span attribute with this value for logical service naming.
+    # - :defer_drain - defaults to false. When true, a quiet/multi block no longer waits at its end for the replies
+    #                  to its requests. The requests are still sent right away; their replies are read (with one noop
+    #                  per server) just before the next non-quiet request to that server, or by
+    #                  #drain_deferred_responses. The tradeoff: an error reply to a quiet request surfaces, and is
+    #                  discarded, at that later point rather than when the block ends. Replies memcached sends for
+    #                  quiet requests (NF, NS, errors) wait on the connection until then, so a client that only makes
+    #                  quiet requests should call #drain_deferred_responses periodically. See "Deferred Draining" in
+    #                  the README.
     #
     def initialize(servers = nil, options = {})
       @normalized_servers = ::Dalli::ServersArgNormalizer.normalize_servers(servers)
@@ -149,6 +157,22 @@ module Dalli
       retry
     end
 
+    ##
+    # Like #get_with_metadata, but wraps the result in a Dalli::CacheResult
+    # instead of returning a Hash.
+    #
+    # Interim adapter (see #1130 / #1151): #get_with_metadata itself keeps
+    # returning a Hash, since changing that return type is a breaking change
+    # reserved for a future major version. This lets callers opt in to the
+    # structured shape today.
+    #
+    # @param key [String] the cache key
+    # @param options [Hash] see #get_with_metadata
+    # @return [Dalli::CacheResult]
+    def get_with_metadata_result(key, options = {})
+      CacheResult.new(get_with_metadata(key, options))
+    end
+
     ##
     # Fetch multiple keys efficiently.
     # If a block is given, yields key/value pairs one at a time.
@@ -228,6 +252,27 @@ module Dalli
       results
     end
 
+    ##
+    # Like #get_multi_with_metadata, but wraps each entry's Hash in a
+    # Dalli::CacheResult instead. Missing keys are still omitted; no block
+    # form.
+    #
+    # Interim adapter (see #1130 / #1151): #get_multi_with_metadata itself
+    # keeps returning a Hash of Hashes, since changing that return type is a
+    # breaking change reserved for a future major version. This lets callers
+    # opt in to the structured shape today.
+    #
+    # Result key order follows #get_multi_with_metadata's own: request order
+    # only when every key lands on the same server, per-server response order
+    # otherwise.
+    #
+    # @param keys [Array<String>] the keys to fetch
+    # @param req_options [Hash, nil] see #get_multi_with_metadata
+    # @return [Hash] key => Dalli::CacheResult
+    def get_multi_with_metadata_result(*keys, req_options: nil)
+      get_multi_with_metadata(*keys, req_options: req_options).transform_values { |hash| CacheResult.new(hash) }
+    end
+
     ##
     # Fetch multiple keys efficiently, including available metadata such as CAS.
     # If a block is given, yields key/data pairs one a time.  Data is an array:
@@ -355,7 +400,8 @@ module Dalli
     # arithmetic (incr, decr), flush and delete operations.  Use of
     # unsupported operations inside a block will raise an error.
     #
-    # Any error replies will be discarded at the end of the block, and
+    # Any error replies will be discarded at the end of the block (or, with
+    # the defer_drain option, before the next non-quiet request), and
     # Dalli client methods invoked inside the block will not
     # have return values
     def quiet
@@ -363,11 +409,25 @@ module Dalli
       Thread.current[::Dalli::QUIET] = true
       yield
     ensure
-      @ring&.pipeline_consume_and_ignore_responses
+      @ring&.pipeline_consume_and_ignore_responses unless @options[:defer_drain]
       Thread.current[::Dalli::QUIET] = old
     end
     alias multi quiet
 
+    ##
+    # Reads and discards the replies still pending from quiet requests, on the
+    # servers that have any. Only needed with the defer_drain option.
+    #
+    # Normally the next non-quiet request to a server drains its pending
+    # replies first. A client that only makes quiet requests (for example, a
+    # worker that only issues quiet deletes) never does, so its replies keep
+    # accumulating on the socket; call this periodically, such as after each
+    # batch or job. It sends nothing to servers with no quiet requests pending.
+    def drain_deferred_responses
+      @ring&.pipeline_consume_and_ignore_responses
+      nil
+    end
+
     def set(key, value, ttl = nil, req_options = nil)
       set_cas(key, value, 0, ttl, req_options)
     end

data/lib/dalli/protocol/base.rb

@@ -18,7 +18,8 @@ module Dalli
 
       def_delegators :@value_marshaller, :serializer, :compressor, :compression_min_size, :compress_by_default?
       def_delegators :@connection_manager, :name, :hostname, :port, :close, :socket_timeout,
-                     :socket_type, :up!, :down!, :write, :reconnect_down_server?, :raise_down_error
+                     :socket_type, :up!, :down!, :write, :reconnect_down_server?, :raise_down_error,
+                     :quiet_responses_pending?
 
       # Delegated by hand rather than with def_delegators because they run on
       # every request, and a plain method call is cheaper than a Forwardable one.
@@ -41,6 +42,7 @@ module Dalli
         warn_uri_credentials(user_creds)
         @options = client_options.merge(user_creds)
         @raw_mode = client_options[:raw]
+        @defer_drain = client_options[:defer_drain] ? true : false
         @value_marshaller = @raw_mode ? StringMarshaller.new(@options) : ValueMarshaller.new(@options)
         @connection_manager = ConnectionManager.new(hostname, port, socket_type, @options)
       end
@@ -58,7 +60,11 @@ module Dalli
         begin
           request_completed = false
           @connection_manager.start_request!
+          drain_deferred_quiet_responses(opkey)
           response = send(opkey, *args)
+          # Quiet requests can leave replies on the socket; remember that so the
+          # end of the quiet block knows this server needs draining
+          @connection_manager.mark_quiet_responses_pending! if quiet? && opkey != :noop
 
           # pipelined_get/pipelined_get_interleaved emit query but don't read the response(s)
           @connection_manager.finish_request! unless %i[pipelined_get pipelined_get_interleaved].include?(opkey)
@@ -217,6 +223,14 @@ module Dalli
       ALLOWED_QUIET_OPS = %i[add replace set delete incr decr append prepend flush noop].freeze
       private_constant :ALLOWED_QUIET_OPS
 
+      # With defer_drain, quiet blocks leave their replies unread. Read them
+      # before any non-quiet request, which reads a reply of its own.
+      def drain_deferred_quiet_responses(opkey)
+        return unless @defer_drain && !quiet? && opkey != :noop && @connection_manager.quiet_responses_pending?
+
+        noop
+      end
+
       def verify_allowed_quiet!(opkey)
         return if ALLOWED_QUIET_OPS.include?(opkey)
 

data/lib/dalli/protocol/connection_manager.rb

@@ -33,6 +33,7 @@ module Dalli
         @socket_type = socket_type
         @options = DEFAULTS.merge(client_options)
         @request_in_progress = false
+        @quiet_responses_pending = false
         @sock = nil
         @pid = nil
 
@@ -54,6 +55,7 @@ module Dalli
         @sock.sync = false # Enable buffered I/O for better performance
         @pid = Process.pid
         @request_in_progress = false
+        @quiet_responses_pending = false
       rescue SystemCallError, *TIMEOUT_ERRORS, EOFError, SocketError => e
         # SocketError = DNS resolution failure
         error_on_request!(e)
@@ -123,6 +125,7 @@ module Dalli
           # @request_in_progress == true.
           @sock = nil
           @pid = nil
+          @quiet_responses_pending = false
           abort_request!
         end
       end
@@ -131,6 +134,21 @@ module Dalli
         !@sock.nil?
       end
 
+      # True when a quiet request was written on this connection and its
+      # replies (memcached only suppresses some of them) haven't been
+      # consumed by a noop yet.
+      def quiet_responses_pending?
+        @quiet_responses_pending
+      end
+
+      def mark_quiet_responses_pending!
+        @quiet_responses_pending = true
+      end
+
+      def clear_quiet_responses_pending!
+        @quiet_responses_pending = false
+      end
+
       def request_in_progress?
         @request_in_progress
       end

data/lib/dalli/protocol/meta.rb

@@ -157,12 +157,17 @@ module Dalli
       def write_storage_req(mode, key, raw_value, ttl = nil, cas = nil, options = {}, quiet: quiet?)
         (value, bitflags) = @value_marshaller.store(key, raw_value, options)
         ttl = TtlSanitizer.sanitize(ttl) if ttl
-        req = RequestFormatter.meta_set(key: key, value: value,
-                                        bitflags: bitflags, cas: cas,
-                                        ttl: ttl, mode: mode, quiet: quiet,
-                                        **routing_token_kwargs(options))
+        # eql?(0): only Integer 0 means "no CAS" here; anything else takes the general path
+        req = if mode == :set && !quiet && (cas.nil? || cas.eql?(0)) && !routing_tokens?(options)
+                RequestFormatter.plain_meta_set(key, value.bytesize, bitflags, ttl)
+              else
+                RequestFormatter.meta_set(key: key, value: value,
+                                          bitflags: bitflags, cas: cas,
+                                          ttl: ttl, mode: mode, quiet: quiet,
+                                          **routing_token_kwargs(options))
+              end
         write("#{req}#{value}#{TERMINATOR}")
-        @connection_manager.flush unless quiet
+        finish_write(quiet)
       end
       # rubocop:enable Metrics/ParameterLists
 
@@ -183,7 +188,7 @@ module Dalli
                                         cas: cas, ttl: ttl, mode: mode, quiet: quiet?,
                                         **routing_token_kwargs(options))
         write("#{req}#{value}#{TERMINATOR}")
-        @connection_manager.flush unless quiet?
+        finish_write(quiet?)
       end
 
       # Delete Commands
@@ -191,10 +196,15 @@ module Dalli
       # `options` supports the meta-delete keys :invalidate, :tombstone_ttl and
       # :drop_value, plus :p_token/:l_token; see Dalli::Client#delete.
       def delete(key, cas, options = nil)
-        req = RequestFormatter.meta_delete(key: key, cas: cas, quiet: quiet?,
-                                           **tombstone_kwargs(options), **routing_token_kwargs(options))
+        # Fast path for a plain delete: no options, no CAS, not quiet
+        req = if options.nil? && (cas.nil? || cas.eql?(0)) && !quiet?
+                RequestFormatter.plain_meta_delete(key)
+              else
+                RequestFormatter.meta_delete(key: key, cas: cas, quiet: quiet?,
+                                             **tombstone_kwargs(options), **routing_token_kwargs(options))
+              end
         write(req)
-        @connection_manager.flush unless quiet?
+        finish_write(quiet?)
         response_processor.meta_delete unless quiet?
       end
 
@@ -225,22 +235,34 @@ module Dalli
         ttl = initial ? TtlSanitizer.sanitize(ttl) : nil # Only set a TTL if we want to set a value on miss
         write(RequestFormatter.meta_arithmetic(key: key, delta: delta, initial: initial, incr: incr, ttl: ttl,
                                                quiet: quiet?, **routing_token_kwargs(options)))
-        @connection_manager.flush unless quiet?
+        finish_write(quiet?)
         response_processor.decr_incr unless quiet?
       end
 
       # Other Commands
       def flush(delay = 0)
         write(RequestFormatter.flush(delay: delay))
-        @connection_manager.flush unless quiet?
+        finish_write(quiet?)
         response_processor.flush unless quiet?
       end
 
       # Noop is a keepalive operation but also used to demarcate the end of a set of pipelined commands.
       # We need to read all the responses at once.
+      # A write that expects a reply is flushed so the reply can be read. A write
+      # in a quiet block stays buffered until the block's noop, unless
+      # defer_drain is on: then no noop follows at the end of the block, so it is
+      # flushed right away. (pipelined_set passes quiet: true outside a quiet
+      # block; its caller flushes with a terminating noop.)
+      def finish_write(quiet)
+        @connection_manager.flush if !quiet || (@defer_drain && quiet?)
+      end
+
       def noop
         write_noop
-        response_processor.consume_all_responses_until_mn
+        result = response_processor.consume_all_responses_until_mn
+        # Everything up to and including the MN has been read
+        @connection_manager.clear_quiet_responses_pending!
+        result
       end
 
       def stats(info = nil)
@@ -313,26 +335,25 @@ module Dalli
 
       def read_multi_get_responses(is_raw)
         hash = {}
-        key_index = is_raw ? 2 : 3
         while (line = @connection_manager.read_line)
           break if line.start_with?('MN')
           next unless line.start_with?('VA ')
 
-          key, value = parse_multi_get_value(line, key_index, is_raw)
+          key, value = parse_multi_get_value(line, is_raw)
           hash[key] = value if key
         end
         hash
       end
 
-      def parse_multi_get_value(line, key_index, is_raw)
-        tokens = line.chomp!(TERMINATOR).split
-        value = @connection_manager.read(tokens[1].to_i + TERMINATOR.bytesize)&.chomp!(TERMINATOR)
-        raw_key = tokens[key_index]
-        return unless raw_key
+      # Reads the "VA <size> [f<flags>] k<key> ..." line in place rather than
+      # splitting it into tokens
+      def parse_multi_get_value(line, is_raw)
+        processor = response_processor
+        value = @connection_manager.read(processor.size_from_va_line(line) + TERMINATOR.bytesize)&.chomp!(TERMINATOR)
+        key = processor.key_from_va_line(line)
+        return unless key
 
-        key = raw_key[1..]
-        key = KeyRegularizer.decode(key) if tokens.include?('b')
-        bitflags = is_raw ? 0 : response_processor.bitflags_from_tokens(tokens)
+        bitflags = is_raw ? 0 : processor.bitflags_from_va_line(line)
         [key, @value_marshaller.retrieve(value, bitflags)]
       end
 

data/lib/dalli/protocol/request_formatter.rb

@@ -63,6 +63,19 @@ module Dalli
           skip_flags ? "mg #{encoded_key(key)} v#{TERMINATOR}" : "mg #{encoded_key(key)} v f#{TERMINATOR}"
         end
 
+        # Fast path for a plain set: mode :set, not quiet, no CAS and no routing
+        # tokens. Produces the same bytes as meta_set(key:, value:, bitflags:, ttl:)
+        # without the keyword-argument handling and incremental string building.
+        def plain_meta_set(key, value_bytesize, bitflags, ttl)
+          flags = bitflags ? " F#{bitflags}" : ''
+          flags += " T#{integer_flag(:ttl, ttl)}" if ttl
+          if KeyRegularizer.required?(key)
+            "ms #{KeyRegularizer.encode(key)} #{value_bytesize} c b#{flags} MS#{TERMINATOR}"
+          else
+            "ms #{key} #{value_bytesize} c#{flags} MS#{TERMINATOR}"
+          end
+        end
+
         # Pass terminate: false to leave off the trailing noop, for callers
         # (the pipelined get) that send it separately.
         def multi_meta_get(keys, skip_flags: false, return_cas: false, terminate: true, p_token: nil, l_token: nil)
@@ -158,6 +171,11 @@ module Dalli
           cmd << TERMINATOR
         end
 
+        # Same bytes as meta_delete(key: key), without the keyword-argument handling
+        def plain_meta_delete(key)
+          "md #{encoded_key(key)}#{TERMINATOR}"
+        end
+
         # Tombstone and routing-token flags apply to every key in the batch;
         # see meta_delete.
         def multi_meta_delete(keys, stale: false, ttl: nil, drop_value: false, p_token: nil, l_token: nil)

data/lib/dalli/protocol/response_processor.rb

@@ -24,7 +24,10 @@ module Dalli
         SERVER_ERROR = 'SERVER_ERROR'
 
         VA_PREFIX = 'VA '
+        HD_PREFIX = 'HD '
         FLAGS_TOKEN_PREFIX = ' f'
+        CAS_TOKEN_PREFIX = ' c'
+        KEY_TOKEN_PREFIX = ' k'
         BYTE_B = 'b'.ord
         BYTE_C = 'c'.ord
         BYTE_F = 'f'.ord
@@ -125,7 +128,13 @@ module Dalli
         end
 
         def meta_set_with_cas
-          tokens = error_on_unexpected!(T_HD_NS_NF_EX)
+          line = read_line
+          # A stored item ("HD c<cas>") is parsed in place rather than split into tokens
+          # Search from just after "HD": the " c" token starts at the space that follows it
+          return flag_from_line(line, CAS_TOKEN_PREFIX, HD.bytesize) if line&.start_with?(HD_PREFIX)
+
+          tokens = line&.split || []
+          raise_unexpected!(tokens) unless T_HD_NS_NF_EX.include?(tokens.first)
           return false unless tokens.first == HD
 
           cas_from_tokens(tokens)
@@ -139,7 +148,13 @@ module Dalli
         end
 
         def meta_delete
-          tokens = error_on_unexpected!(T_HD_NF_EX)
+          line = read_line
+          # The two common replies, compared as whole lines instead of split into tokens
+          return true if line == HD
+          return false if line == NF
+
+          tokens = line&.split || []
+          raise_unexpected!(tokens) unless T_HD_NF_EX.include?(tokens.first)
           tokens.first == HD
         end
 
@@ -217,6 +232,11 @@ module Dalli
           term_idx = buf.byteindex(TERMINATOR, offset)
           return [0] unless term_idx
 
+          if buf.byteslice(offset, VA_PREFIX.bytesize) == VA_PREFIX
+            response = va_response_from_buffer(buf, offset, term_idx)
+            return response if response
+          end
+
           header = buf.byteslice(offset, term_idx - offset)
           tokens = header.split
           header_len = header.bytesize + TERMINATOR.length
@@ -258,6 +278,43 @@ module Dalli
           [tokens.first == VA, flag_int(cas, 'c'), key, value, resp_size]
         end
 
+        # getk_response_from_buffer for a "VA ..." header, read in place
+        # without allocating the header, its tokens or the flag strings.
+        # Gives the same result as the token path. Returns nil to fall back to
+        # it when the header has no s flag or a zero size.
+        def va_response_from_buffer(buf, offset, term_idx)
+          size = bitflags = cas = key = nil
+          base64 = false
+          pos = buf.byteindex(' ', offset + VA_PREFIX.bytesize)
+          while pos && pos < term_idx
+            start = pos + 1
+            pos = buf.byteindex(' ', start)
+            stop = pos && pos < term_idx ? pos : term_idx
+            case buf.getbyte(start)
+            when BYTE_S then size ||= flag_int_at(buf, start, stop)
+            when BYTE_F then bitflags ||= flag_int_at(buf, start, stop)
+            when BYTE_C then cas ||= flag_int_at(buf, start, stop)
+            when BYTE_K then key ||= buf.byteslice(start + 1, stop - start - 1)
+            when BYTE_B then base64 ||= stop == start + 1
+            end
+          end
+          return nil if size.nil? || size.zero?
+
+          header_len = term_idx - offset + TERMINATOR.length
+          resp_size = header_len + size + TERMINATOR.length
+          return [0] unless buf.bytesize >= offset + resp_size
+
+          value = @value_marshaller.retrieve(buf.byteslice(offset + header_len, size), bitflags || 0)
+          key = KeyRegularizer.decode(key) if base64 && key
+          [true, cas || 0, key || 0, value, resp_size]
+        end
+
+        # Integer value of a flag's token in buf, after its one-byte prefix,
+        # as flag_int would read it
+        def flag_int_at(buf, start, stop)
+          buf.byteslice(start + 1, stop - start - 1).to_i
+        end
+
         # Integer value of a flag token such as "f123", or 0 when absent.
         # Strips the prefix in place, like value_from_tokens.
         def flag_int(token, flag)
@@ -289,8 +346,29 @@ module Dalli
         end
 
         def bitflags_from_va_line(line)
-          idx = line.index(FLAGS_TOKEN_PREFIX, VA_PREFIX.bytesize)
-          idx ? line.byteslice(idx + FLAGS_TOKEN_PREFIX.bytesize, line.bytesize).to_i : 0
+          flag_from_line(line, FLAGS_TOKEN_PREFIX, VA_PREFIX.bytesize)
+        end
+
+        # The key from a VA line's k flag, decoded when the line has the b flag,
+        # or nil when there is no k flag
+        def key_from_va_line(line)
+          idx = line.index(KEY_TOKEN_PREFIX, VA_PREFIX.bytesize)
+          return unless idx
+
+          start = idx + KEY_TOKEN_PREFIX.bytesize
+          stop = line.index(' ', start) || line.index(TERMINATOR, start) || line.bytesize
+          key = line.byteslice(start, stop - start)
+          base64_flag_in_line?(line) ? KeyRegularizer.decode(key) : key
+        end
+
+        def base64_flag_in_line?(line)
+          line.include?(' b ') || line.end_with?(" b#{TERMINATOR}", ' b')
+        end
+
+        # Integer value of the first " <flag><digits>" token at or after start, or 0
+        def flag_from_line(line, token_prefix, start)
+          idx = line.index(token_prefix, start)
+          idx ? line.byteslice(idx + token_prefix.bytesize, line.bytesize).to_i : 0
         end
 
         def cas_from_tokens(tokens)

data/lib/dalli/ring.rb

@@ -36,9 +36,10 @@ module Dalli
 
     def continuum=(entries)
       @continuum = entries
-      # Plain integers, so the binary search in server_for_hash_key needn't
-      # call Entry#value at each step
+      # Plain integers, so the lookup in server_for_hash_key needn't call
+      # Entry#value at each step
       @continuum_values = entries&.map(&:value)
+      build_buckets
     end
 
     # alive_cache (optional) remembers each server's alive? result, so a
@@ -55,15 +56,17 @@ module Dalli
       raise Dalli::RingError, 'No server available'
     end
 
+    # Tries the key's own server first, then (with failover) up to 19
+    # rehashes of "<try><key>". The first try is outside the loop because it
+    # is almost always the one that's used.
     def server_from_continuum(key, alive_cache = nil)
-      hkey = hash_for(key)
-      20.times do |try|
-        server = server_for_hash_key(hkey)
+      server = server_for_hash_key(hash_for(key))
+      return server if server_alive?(server, alive_cache)
+      return nil unless @failover
 
+      19.times do |try|
+        server = server_for_hash_key(hash_for("#{try}#{key}"))
         return server if server_alive?(server, alive_cache)
-        break unless @failover
-
-        hkey = hash_for("#{try}#{key}")
       end
       nil
     end
@@ -87,8 +90,13 @@ module Dalli
       end
     end
 
+    # Drains the replies left by quiet requests. Only servers that were sent a
+    # quiet request need it, so the others (and servers never connected) are
+    # skipped rather than each costing a noop round trip.
     def pipeline_consume_and_ignore_responses
       @servers.each do |s|
+        next unless s.quiet_responses_pending?
+
         s.request(:noop)
       rescue Dalli::NetworkError
         # Ignore this error, as it indicates the socket is unavailable
@@ -121,22 +129,41 @@ module Dalli
     def server_alive?(server, alive_cache)
       return server.alive? unless alive_cache
 
-      alive_cache.fetch(server) { alive_cache[server] = server.alive? }
+      alive = alive_cache[server]
+      alive.nil? ? (alive_cache[server] = server.alive?) : alive
     end
 
     def entry_count_for(server, total_servers, total_weight)
       ((total_servers * POINTS_PER_SERVER * server.weight) / Float(total_weight)).floor
     end
 
-    def server_for_hash_key(hash_key)
-      # Find the closest index in the Ring with value <= the given value
-      entryidx = @continuum_values.bsearch_index { |value| value > hash_key }
-      if entryidx.nil?
-        entryidx = @continuum.size - 1
-      else
-        entryidx -= 1
+    # Hash values are 32 bits. Splitting that range into about one bucket per
+    # continuum entry, and noting the first entry at or after each bucket's
+    # start, lets server_for_hash_key skip the binary search: it starts at the
+    # key's bucket and steps over the few entries before the key's hash.
+    def build_buckets
+      return unless @continuum_values
+
+      bits = [@continuum_values.size.bit_length, 1].max
+      @bucket_shift = 32 - bits
+      @bucket_first = Array.new(1 << bits)
+      idx = 0
+      size = @continuum_values.size
+      @bucket_first.each_index do |bucket|
+        floor = bucket << @bucket_shift
+        idx += 1 while idx < size && @continuum_values[idx] < floor
+        @bucket_first[bucket] = idx
       end
-      @continuum[entryidx].server
+    end
+
+    def server_for_hash_key(hash_key)
+      # Find the last entry with value <= hash_key, wrapping around to the
+      # last entry when every value is greater (idx - 1 is then -1)
+      idx = @bucket_first[hash_key >> @bucket_shift]
+      values = @continuum_values
+      size = values.size
+      idx += 1 while idx < size && values[idx] <= hash_key
+      @continuum[idx - 1].server
     end
 
     def build_continuum(servers)

data/lib/dalli/socket.rb

@@ -12,22 +12,29 @@ module Dalli
     # Common methods for all socket implementations.
     ##
     module InstanceMethods
+      READ_CHUNK_SIZE = 8196
+
+      # Reads whatever is available without blocking. A read that returns less
+      # than a full chunk means the socket was drained, so we stop there rather
+      # than spend another read_nonblock call just to get :wait_readable back.
+      # Callers keep waiting on IO.select for anything that arrives later.
       def read_available(reusable_buffer = nil)
         if reusable_buffer
-          value = read_nonblock(8196, reusable_buffer, exception: false)
+          value = read_nonblock(READ_CHUNK_SIZE, reusable_buffer, exception: false)
           case value
           when :wait_writable, :wait_readable
             return reusable_buffer.clear
           when nil
             raise Errno::ECONNRESET, "Connection reset: #{logged_options.inspect}"
           end
+          return value if drained_after?(value)
         else
           value = ''.b
         end
 
         buffer = ''.b
         loop do
-          result = read_nonblock(8196, buffer, exception: false)
+          result = read_nonblock(READ_CHUNK_SIZE, buffer, exception: false)
           case result
           when :wait_writable, :wait_readable
             buffer.clear
@@ -36,10 +43,21 @@ module Dalli
             raise Errno::ECONNRESET, "Connection reset: #{logged_options.inspect}"
           else
             value << result
+            return value if drained_after?(result)
           end
         end
       end
 
+      # True when a read came back short and nothing is left buffered in-process.
+      def drained_after?(chunk)
+        chunk.bytesize < READ_CHUNK_SIZE && !buffered_data?
+      end
+
+      # Plain sockets keep nothing buffered in-process. See SSLSocket.
+      def buffered_data?
+        false
+      end
+
       FILTERED_OUT_OPTIONS = %i[username password].freeze
       def logged_options
         options.except(*FILTERED_OUT_OPTIONS)
@@ -83,6 +101,13 @@ module Dalli
         io.options
       end
 
+      # OpenSSL can hold decrypted bytes that IO.select won't report as readable.
+      # With OpenSSL's default (no read-ahead) a short read has already used up
+      # the current record, so this is normally 0; it guards read-ahead setups.
+      def buffered_data?
+        pending.positive?
+      end
+
       unless method_defined?(:wait_readable)
         def wait_readable(timeout = nil)
           to_io.wait_readable(timeout)

data/lib/dalli/version.rb

@@ -1,7 +1,7 @@
 # frozen_string_literal: true
 
 module Dalli
-  VERSION = '5.1.2'
+  VERSION = '5.2.0'
 
   MIN_SUPPORTED_MEMCACHED_VERSION = '1.6.27'
 end

data/lib/dalli.rb

@@ -63,6 +63,7 @@ require_relative 'dalli/instrumentation'
 
 require_relative 'dalli/flags'
 require_relative 'dalli/compressor'
+require_relative 'dalli/cache_result'
 require_relative 'dalli/client'
 require_relative 'dalli/key_manager'
 require_relative 'dalli/pipelined_getter'

metadata

@@ -1,7 +1,7 @@
 --- !ruby/object:Gem::Specification
 name: dalli
 version: !ruby/object:Gem::Version
-  version: 5.1.2
+  version: 5.2.0
 platform: ruby
 authors:
 - Peter M. Goldstein
@@ -37,6 +37,7 @@ files:
 - LICENSE
 - README.md
 - lib/dalli.rb
+- lib/dalli/cache_result.rb
 - lib/dalli/cas/client.rb
 - lib/dalli/client.rb
 - lib/dalli/compressor.rb