dalli 2.7.6 → 2.7.11

checksums.yaml

@@ -1,7 +1,7 @@
 ---
-SHA1:
-  metadata.gz: aa780eecd8063c6371f755a081c672bee694c596
-  data.tar.gz: f044315c06b99f3ff10da4c4a37a4b77cce359da
+SHA256:
+  metadata.gz: 4bc4fe873fe9319c9297219b7908d576470551d0e011d1299e54ebc162256a68
+  data.tar.gz: b60393783d05152ad67aa12298f6867127f973fbf07cd2d0dc90ab96b1d7971f
 SHA512:
-  metadata.gz: 962196b3d7c2a75b928f32e896106ead0316b6d2614f1176041a3751c82816e5669a70d9d293ef1762df190bade229955b20c5673562600575774b3bfed6c978
-  data.tar.gz: c890f77ce55953a3f0fe47629ddb1eda40ece7b58628c7168e45c3733c5120e72c7dba992f6b15267ea5025283bf8f5c2e213648c65e6a87131219bfe318f5a1
+  metadata.gz: 1348b528243f253c111ad74e17a0ba20d8d14eb29d4f41e15f60bb2c87505c6d09404217c868f1623bd68f1ad70a644efd09fd1acca0ba2a1d0b6395387967ae
+  data.tar.gz: 108c5477c6399ef838a8df14e4878509ec5f7c22c33d012c19a9deb635901e9251a52c392c080fd6eb65dfb54e8084ee70cf6668a6914fd1336120aabb19b4f9

data/Gemfile

@@ -2,4 +2,14 @@ source 'https://rubygems.org'
 
 gemspec
 
+gem 'appraisal'
 gem 'kgio', :platform => :mri
+gem 'rails'
+
+group :test do
+  gem 'minitest'
+  gem 'mocha'
+  gem 'rake'
+  gem 'connection_pool'
+  gem 'simplecov'
+end

data/History.md

@@ -1,6 +1,44 @@
 Dalli Changelog
 =====================
 
+2.7.11
+==========
+- DEPRECATION: :dalli_store will be removed in Dalli 3.0.
+  Use Rails' official :mem_cache_store instead.
+  https://guides.rubyonrails.org/caching_with_rails.html
+- Add new `digest_class` option to Dalli::Client [#724]
+- Don't treat NameError as a network error [#728]
+- Handle nested comma separated server strings (sambostock)
+
+2.7.10
+==========
+- Revert frozen string change (schneems)
+- Advertise supports_cached_versioning? in DalliStore (schneems)
+- Better detection of fork support, to allow specs to run under Truffle Ruby (deepj)
+- Update logging for over max size to log as error (aeroastro)
+
+2.7.9
+==========
+- Fix behavior for Rails 5.2+ cache_versioning (GriwMF)
+- Ensure fetch provides the key to the fallback block as an argument (0exp)
+- Assorted performance improvements (schneems)
+
+2.7.8
+==========
+- Rails 5.2 compatibility (pbougie)
+- Fix Session Cache compatibility (pixeltrix)
+
+2.7.7
+==========
+- Support large cache keys on fetch multi (sobrinho)
+- Not found checks no longer trigger the result's equality method (dannyfallon)
+- Use SVG build badges (olleolleolle)
+- Travis updates (junaruga, tiarly, petergoldstein)
+- Update default down_retry_delay (jaredhales)
+- Close kgio socket after IO.select timeouts
+- Documentation updates (tipair)
+- Instrument DalliStore errors with instrument_errors configuration option. (btatnall)
+
 2.7.6
 ==========
 - Rails 5.0.0.beta2 compatibility (yui-knk, petergoldstein)

data/README.md

@@ -1,4 +1,4 @@
-Dalli [![Build Status](https://secure.travis-ci.org/petergoldstein/dalli.png)](http://travis-ci.org/petergoldstein/dalli) [![Dependency Status](https://gemnasium.com/petergoldstein/dalli.png)](https://gemnasium.com/petergoldstein/dalli) [![Code Climate](https://codeclimate.com/github/petergoldstein/dalli.png)](https://codeclimate.com/github/petergoldstein/dalli)
+Dalli [![Build Status](https://secure.travis-ci.org/petergoldstein/dalli.svg)](http://travis-ci.org/petergoldstein/dalli)
 =====
 
 Dalli is a high performance pure Ruby client for accessing memcached servers.  It works with memcached 1.4+ only as it uses the newer binary protocol.  It should be considered a replacement for the memcache-client gem.
@@ -9,26 +9,6 @@ The name is a variant of Salvador Dali for his famous painting [The Persistence
 
 Dalli's initial development was sponsored by [CouchBase](http://www.couchbase.com/).  Many thanks to them!
 
-
-Design
-------------
-
-Mike Perham decided to write Dalli after maintaining memcache-client for two years for a few specific reasons:
-
- 0. The code is mostly old and gross.  The bulk of the code is a single 1000 line .rb file.
- 1. It has a lot of options that are infrequently used which complicate the codebase.
- 2. The implementation has no single point to attach monitoring hooks.
- 3. Uses the old text protocol, which hurts raw performance.
-
-So a few notes.  Dalli:
-
- 0. uses the exact same algorithm to choose a server so existing memcached clusters with TBs of data will work identically to memcache-client.
- 1. is approximately 20% faster than memcache-client (which itself was heavily optimized) in Ruby 1.9.2.
- 2. contains explicit "chokepoint" methods which handle all requests; these can be hooked into by monitoring tools (NewRelic, Rack::Bug, etc) to track memcached usage.
- 3. supports SASL for use in managed environments, e.g. Heroku.
- 4. provides proper failover with recovery and adjustable timeouts
-
-
 Supported Ruby versions and implementations
 ------------------------------------------------
 
@@ -66,11 +46,9 @@ dc.set('abc', 123)
 value = dc.get('abc')
 ```
 
-The test suite requires memcached 1.4.3+ with SASL enabled (brew install memcached --enable-sasl ; mv /usr/bin/memcached /usr/bin/memcached.old).  Currently only supports the PLAIN mechanism.
-
-Dalli has no runtime dependencies and never will.  You can optionally install the 'kgio' gem to
-give Dalli a 20-30% performance boost.
+The test suite requires memcached 1.4.3+ with SASL enabled (`brew install memcached --enable-sasl ; mv /usr/bin/memcached /usr/bin/memcached.old`).  Currently only supports the PLAIN mechanism.
 
+Dalli has no runtime dependencies.
 
 Usage with Rails 3.x and 4.x
 ---------------------------
@@ -90,10 +68,12 @@ config.cache_store = :dalli_store
 Here's a more comprehensive example that sets a reasonable default for maximum cache entry lifetime (one day), enables compression for large values and namespaces all entries for this rails app.  Remove the namespace if you have multiple apps which share cached values.
 
 ```ruby
-config.cache_store = :dalli_store, 'cache-1.example.com', 'cache-2.example.com',
+config.cache_store = :dalli_store, 'cache-1.example.com', 'cache-2.example.com:11211:2',
   { :namespace => NAME_OF_RAILS_APP, :expires_in => 1.day, :compress => true }
 ```
 
+You can specify a port and a weight by appending to the server name. You may wish to increase the weight of a server with more memory configured.  (e.g. to specify port 11211 and a weight of 2, append `:11211:2` ) 
+
 If your servers are specified in `ENV["MEMCACHE_SERVERS"]` (e.g. on Heroku when using a third-party hosted addon), simply provide `nil` for the servers:
 
 ```ruby
@@ -147,6 +127,8 @@ end
 Configuration
 ------------------------
 
+**servers**: An Array of "host:port:weight" where weight allows you to distribute cache unevenly.
+
 Dalli::Client accepts the following options. All times are in seconds.
 
 **expires_in**: Global default for key TTL.  Default is 0, which means no expiry.
@@ -155,7 +137,7 @@ Dalli::Client accepts the following options. All times are in seconds.
 
 **failover**: Boolean, if true Dalli will failover to another server if the main server for a key is down.  Default is true.
 
-**threadsafe**: Boolean.  If true Dalli ensures that only one thread is using a socket at a given time.  Default is true.  Set to false at your own peril.
+**threadsafe**: Boolean.  If true Dalli ensures that only one thread is using a socket at a given time. Default is true. You can set to false if you are using the Client within a thread-safe connection pool.
 
 **serializer**: The serializer to use for objects being stored (ex. JSON).
 Default is Marshal.
@@ -178,10 +160,12 @@ If serving compressed data using nginx's HttpMemcachedModule, set `memcached_gzi
 
 **socket_failure_delay**: Before retrying a socket operation, the process sleeps for this amount of time. Default is 0.01.  Set to nil for no delay.
 
-**down_retry_delay**: When a server has been marked down due to many failures, the server will be checked again for being alive only after this amount of time. Don't set this value too low, otherwise each request which tries the failed server might hang for the maximum **socket_timeout**. Default is 1 second.
+**down_retry_delay**: When a server has been marked down due to many failures, the server will be checked again for being alive only after this amount of time. Don't set this value too low, otherwise each request which tries the failed server might hang for the maximum **socket_timeout**. Default is 60 seconds.
 
 **value_max_bytes**: The maximum size of a value in memcached.  Defaults to 1MB, this can be increased with memcached's -I parameter.  You must also configure Dalli to allow the larger size here.
 
+**error_when_over_max_size**: Boolean. If true, Dalli will throw a Dalli::ValueOverMaxSize exception when trying to store data larger than **value_max_bytes**. Defaults to false, meaning only a warning is logged.
+
 **username**: The username to use for authenticating this client instance against a SASL-enabled memcached server.  Heroku users should not need to use this normally.
 
 **password**: The password to use for authenticating this client instance against a SASL-enabled memcached server.  Heroku users should not need to use this normally.
@@ -192,10 +176,14 @@ If serving compressed data using nginx's HttpMemcachedModule, set `memcached_gzi
 
 **cache_nils**: Boolean. If true Dalli will not treat cached `nil` values as 'not found' for `#fetch` operations. Default is false.
 
+**raise_errors**: Boolean. When true DalliStore will reraise Dalli:DalliError instead swallowing the error. Default is false.
+
+**instrument_errors**: Boolean. When true DalliStore will send notification of Dalli::DalliError via a 'cache_error.active_support' event. Default is false.
+
 Features and Changes
 ------------------------
 
-By default, Dalli is thread-safe.  Disable thread-safety at your own peril.
+By default, Dalli is thread-safe. Disable thread-safety at your own peril.
 
 Dalli does not need anything special in Unicorn/Passenger since 2.0.4.
 It will detect sockets shared with child processes and gracefully reopen the
@@ -208,7 +196,7 @@ Note that Dalli does not require ActiveSupport or Rails.  You can safely use it
 Helping Out
 -------------
 
-If you have a fix you wish to provide, please fork the code, fix in your local project and then send a pull request on github.  Please ensure that you include a test which verifies your fix and update History.md with a one sentence description of your fix so you get credit as a contributor.
+If you have a fix you wish to provide, please fork the code, fix in your local project and then send a pull request on github.  Please ensure that you include a test which verifies your fix and update `History.md` with a one sentence description of your fix so you get credit as a contributor.
 
 We're not accepting new compressors. They are trivial to add in an initializer. See #385 (LZ4), #406 (Snappy)
 

data/lib/action_dispatch/middleware/session/dalli_store.rb

@@ -1,3 +1,4 @@
+# frozen_string_literal: true
 require 'active_support/cache'
 require 'action_dispatch/middleware/session/abstract_store'
 require 'dalli'

data/lib/active_support/cache/dalli_store.rb

@@ -1,4 +1,5 @@
 # encoding: ascii
+# frozen_string_literal: true
 require 'dalli'
 
 module ActiveSupport
@@ -8,6 +9,10 @@ module ActiveSupport
       attr_reader :silence, :options
       alias_method :silence?, :silence
 
+      def self.supports_cache_versioning?
+        true
+      end
+
       # Silence the logger.
       def silence!
         @silence = true
@@ -45,6 +50,11 @@ module ActiveSupport
       # wish to use pool support.
       #
       def initialize(*addresses)
+        puts <<-EOS
+DEPRECATION: :dalli_store will be removed in Dalli 3.0.
+Please use Rails' official :mem_cache_store instead.
+https://guides.rubyonrails.org/caching_with_rails.html
+EOS
         addresses = addresses.flatten
         options = addresses.extract_options!
         @options = options.dup
@@ -98,24 +108,24 @@ module ActiveSupport
         if block_given?
           entry = not_found
           unless options[:force]
-            entry = instrument(:read, namespaced_name, options) do |payload|
+            entry = instrument_with_log(:read, namespaced_name, options) do |payload|
               read_entry(namespaced_name, options).tap do |result|
                 if payload
                   payload[:super_operation] = :fetch
-                  payload[:hit] = result != not_found
+                  payload[:hit] = not_found != result
                 end
               end
             end
           end
 
-          if entry == not_found
-            result = instrument(:generate, namespaced_name, options) do |payload|
-              yield
+          if not_found == entry
+            result = instrument_with_log(:generate, namespaced_name, options) do |payload|
+              yield(name)
             end
             write(name, result, options)
             result
           else
-            instrument(:fetch_hit, namespaced_name, options) { |payload| }
+            instrument_with_log(:fetch_hit, namespaced_name, options) { |payload| }
             entry
           end
         else
@@ -127,7 +137,7 @@ module ActiveSupport
         options ||= {}
         name = namespaced_key(name, options)
 
-        instrument(:read, name, options) do |payload|
+        instrument_with_log(:read, name, options) do |payload|
           entry = read_entry(name, options)
           payload[:hit] = !entry.nil? if payload
           entry
@@ -138,7 +148,7 @@ module ActiveSupport
         options ||= {}
         name = namespaced_key(name, options)
 
-        instrument(:write, name, options) do |payload|
+        instrument_with_log(:write, name, options) do |payload|
           with do |connection|
             options = options.merge(:connection => connection)
             write_entry(name, value, options)
@@ -158,7 +168,7 @@ module ActiveSupport
         options ||= {}
         name = namespaced_key(name, options)
 
-        instrument(:delete, name, options) do |payload|
+        instrument_with_log(:delete, name, options) do |payload|
           delete_entry(name, options)
         end
       end
@@ -168,7 +178,7 @@ module ActiveSupport
       def read_multi(*names)
         options  = names.extract_options!
         mapping = names.inject({}) { |memo, name| memo[namespaced_key(name, options)] = name; memo }
-        instrument(:read_multi, mapping.keys) do
+        instrument_with_log(:read_multi, mapping.keys) do
           results = {}
           if local_cache
             mapping.each_key do |key|
@@ -199,7 +209,7 @@ module ActiveSupport
         options = names.extract_options!
         mapping = names.inject({}) { |memo, name| memo[namespaced_key(name, options)] = name; memo }
 
-        instrument(:fetch_multi, mapping.keys) do
+        instrument_with_log(:fetch_multi, mapping.keys) do
           with do |connection|
             results = connection.get_multi(mapping.keys)
 
@@ -230,11 +240,12 @@ module ActiveSupport
         name = namespaced_key(name, options)
         initial = options.has_key?(:initial) ? options[:initial] : amount
         expires_in = options[:expires_in]
-        instrument(:increment, name, :amount => amount) do
+        instrument_with_log(:increment, name, :amount => amount) do
           with { |c| c.incr(name, amount, expires_in, initial) }
         end
       rescue Dalli::DalliError => e
-        logger.error("DalliError: #{e.message}") if logger
+        log_dalli_error(e)
+        instrument_error(e) if instrument_errors?
         raise if raise_errors?
         nil
       end
@@ -249,11 +260,12 @@ module ActiveSupport
         name = namespaced_key(name, options)
         initial = options.has_key?(:initial) ? options[:initial] : 0
         expires_in = options[:expires_in]
-        instrument(:decrement, name, :amount => amount) do
+        instrument_with_log(:decrement, name, :amount => amount) do
           with { |c| c.decr(name, amount, expires_in, initial) }
         end
       rescue Dalli::DalliError => e
-        logger.error("DalliError: #{e.message}") if logger
+        log_dalli_error(e)
+        instrument_error(e) if instrument_errors?
         raise if raise_errors?
         nil
       end
@@ -261,11 +273,12 @@ module ActiveSupport
       # Clear the entire cache on all memcached servers. This method should
       # be used with care when using a shared cache.
       def clear(options=nil)
-        instrument(:clear, 'flushing all keys') do
+        instrument_with_log(:clear, 'flushing all keys') do
           with { |c| c.flush_all }
         end
       rescue Dalli::DalliError => e
-        logger.error("DalliError: #{e.message}") if logger
+        log_dalli_error(e)
+        instrument_error(e) if instrument_errors?
         raise if raise_errors?
         nil
       end
@@ -299,7 +312,8 @@ module ActiveSupport
         # NB Backwards data compatibility, to be removed at some point
         entry.is_a?(ActiveSupport::Cache::Entry) ? entry.value : entry
       rescue Dalli::DalliError => e
-        logger.error("DalliError: #{e.message}") if logger
+        log_dalli_error(e)
+        instrument_error(e) if instrument_errors?
         raise if raise_errors?
         nil
       end
@@ -313,7 +327,8 @@ module ActiveSupport
         connection = options.delete(:connection)
         connection.send(method, key, value, expires_in, options)
       rescue Dalli::DalliError => e
-        logger.error("DalliError: #{e.message}") if logger
+        log_dalli_error(e)
+        instrument_error(e) if instrument_errors?
         raise if raise_errors?
         false
       end
@@ -322,7 +337,8 @@ module ActiveSupport
       def delete_entry(key, options) # :nodoc:
         with { |c| c.delete(key) }
       rescue Dalli::DalliError => e
-        logger.error("DalliError: #{e.message}") if logger
+        log_dalli_error(e)
+        instrument_error(e) if instrument_errors?
         raise if raise_errors?
         false
       end
@@ -330,18 +346,22 @@ module ActiveSupport
       private
 
       def namespaced_key(key, options)
+        digest_class = @options[:digest_class] || ::Digest::MD5
         key = expanded_key(key)
         namespace = options[:namespace] if options
         prefix = namespace.is_a?(Proc) ? namespace.call : namespace
         key = "#{prefix}:#{key}" if prefix
+        key = "#{key[0, 213]}:md5:#{digest_class.hexdigest(key)}" if key && key.size > 250
         key
       end
       alias :normalize_key :namespaced_key
 
-      # Expand key to be a consistent string value. Invoke +cache_key+ if
-      # object responds to +cache_key+. Otherwise, to_param method will be
-      # called. If the key is a Hash, then keys will be sorted alphabetically.
+      # Expand key to be a consistent string value. Invokes +cache_key_with_version+
+      # first to support Rails 5.2 cache versioning.
+      # Invoke +cache_key+ if object responds to +cache_key+. Otherwise, to_param method
+      # will be called. If the key is a Hash, then keys will be sorted alphabetically.
       def expanded_key(key) # :nodoc:
+        return key.cache_key_with_version.to_s if key.respond_to?(:cache_key_with_version)
         return key.cache_key.to_s if key.respond_to?(:cache_key)
 
         case key
@@ -363,12 +383,26 @@ module ActiveSupport
         key
       end
 
-      def instrument(operation, key, options=nil)
+      def log_dalli_error(error)
+        logger.error("DalliError: #{error.message}") if logger
+      end
+
+      def instrument_with_log(operation, key, options=nil)
         log(operation, key, options)
 
         payload = { :key => key }
         payload.merge!(options) if options.is_a?(Hash)
-        ActiveSupport::Notifications.instrument("cache_#{operation}.active_support", payload){ yield(payload) }
+        instrument(operation, payload) { |p| yield(p) }
+      end
+
+      def instrument_error(error)
+        instrument(:error, { :key => 'DalliError', :message => error.message })
+      end
+
+      def instrument(operation, payload)
+        ActiveSupport::Notifications.instrument("cache_#{operation}.active_support", payload) do
+          yield(payload) if block_given?
+        end
       end
 
       def log(operation, key, options=nil)
@@ -380,12 +414,16 @@ module ActiveSupport
         !!@options[:raise_errors]
       end
 
+      def instrument_errors?
+        !!@options[:instrument_errors]
+      end
+
       # Make sure LocalCache is giving raw values, not `Entry`s, and
       # respect `raw` option.
       module LocalCacheEntryUnwrapAndRaw # :nodoc:
         protected
           def read_entry(key, options)
-            retval = super
+            retval = super(key, **options)
             if retval.is_a? ActiveSupport::Cache::Entry
               # Must have come from LocalStore, unwrap it
               if options[:raw]

data/lib/dalli.rb

@@ -1,3 +1,4 @@
+# frozen_string_literal: true
 require 'dalli/compressor'
 require 'dalli/client'
 require 'dalli/ring'
@@ -18,6 +19,8 @@ module Dalli
   class MarshalError < DalliError; end
   # application error in marshalling deserialization or decompression
   class UnmarshalError < DalliError; end
+  # payload too big for memcached
+  class ValueOverMaxSize < DalliError; end
 
   def self.logger
     @logger ||= (rails_logger || default_logger)

data/lib/dalli/cas/client.rb

@@ -1,3 +1,4 @@
+# frozen_string_literal: true
 require 'dalli/client'
 
 module Dalli

data/lib/dalli/client.rb

@@ -1,3 +1,4 @@
+# frozen_string_literal: true
 require 'digest/md5'
 require 'set'
 
@@ -28,6 +29,7 @@ module Dalli
     # - :serializer - defaults to Marshal
     # - :compressor - defaults to zlib
     # - :cache_nils - defaults to false, if true Dalli will not treat cached nil values as 'not found' for #fetch operations.
+    # - :digest_class - defaults to Digest::MD5, allows you to pass in an object that responds to the hexdigest method, useful for injecting a FIPS compliant hash object.
     #
     def initialize(servers=nil, options={})
       @servers = normalize_servers(servers || ENV["MEMCACHE_SERVERS"] || '127.0.0.1:11211')
@@ -63,7 +65,10 @@ module Dalli
     # If a block is given, yields key/value pairs one at a time.
     # Otherwise returns a hash of { 'key' => 'value', 'key2' => 'value1' }
     def get_multi(*keys)
-      return {} if keys.flatten.compact.empty?
+      check_keys = keys.flatten
+      check_keys.compact!
+
+      return {} if check_keys.empty?
       if block_given?
         get_multi_yielder(keys) {|k, data| yield k, data.first}
       else
@@ -291,7 +296,9 @@ module Dalli
     end
 
     def mapped_keys(keys)
-      keys.flatten.map {|a| validate_key(a.to_s)}
+      keys_array = keys.flatten
+      keys_array.map! { |a| validate_key(a.to_s) }
+      keys_array
     end
 
     def make_multi_get_requests(groups)
@@ -325,13 +332,15 @@ module Dalli
 
     ##
     # Normalizes the argument into an array of servers.
-    # If the argument is a string, it's expected that the URIs are comma separated e.g.
+    # If the argument is a string, or an array containing strings, it's expected that the URIs are comma separated e.g.
     # "memcache1.example.com:11211,memcache2.example.com:11211,memcache3.example.com:11211"
     def normalize_servers(servers)
-      if servers.is_a? String
-        return servers.split(",")
-      else
-        return servers
+      Array(servers).flat_map do |server|
+        if server.is_a? String
+          server.split(",")
+        else
+          server
+        end
       end
     end
 
@@ -372,8 +381,9 @@ module Dalli
       raise ArgumentError, "key cannot be blank" if !key || key.length == 0
       key = key_with_namespace(key)
       if key.length > 250
+        digest_class = @options[:digest_class] || ::Digest::MD5
         max_length_before_namespace = 212 - (namespace || '').size
-        key = "#{key[0, max_length_before_namespace]}:md5:#{Digest::MD5.hexdigest(key)}"
+        key = "#{key[0, max_length_before_namespace]}:md5:#{digest_class.hexdigest(key)}"
       end
       return key
     end
@@ -401,6 +411,9 @@ module Dalli
       rescue NoMethodError
         raise ArgumentError, "cannot convert :expires_in => #{opts[:expires_in].inspect} to an integer"
       end
+      if opts[:digest_class] && !opts[:digest_class].respond_to?(:hexdigest)
+        raise ArgumentError, "The digest_class object must respond to the hexdigest method"
+      end
       opts
     end
 

data/lib/dalli/compressor.rb

@@ -1,3 +1,4 @@
+# frozen_string_literal: true
 require 'zlib'
 require 'stringio'
 
@@ -14,7 +15,7 @@ module Dalli
 
   class GzipCompressor
     def self.compress(data)
-      io = StringIO.new("w")
+      io = StringIO.new(String.new(""), "w")
       gz = Zlib::GzipWriter.new(io)
       gz.write(data)
       gz.close

data/lib/dalli/options.rb

@@ -1,3 +1,4 @@
+# frozen_string_literal: true
 require 'thread'
 require 'monitor'
 

data/lib/dalli/railtie.rb

@@ -1,3 +1,4 @@
+# frozen_string_literal: true
 module Dalli
   class Railtie < ::Rails::Railtie
     config.before_configuration do

data/lib/dalli/ring.rb

@@ -1,3 +1,4 @@
+# frozen_string_literal: true
 require 'digest/sha1'
 require 'zlib'
 
@@ -20,7 +21,7 @@ module Dalli
             continuum << Dalli::Ring::Entry.new(value, server)
           end
         end
-        @continuum = continuum.sort { |a, b| a.value <=> b.value }
+        @continuum = continuum.sort_by(&:value)
       end
 
       threadsafe! unless options[:threadsafe] == false
@@ -110,7 +111,6 @@ module Dalli
       def binary_search(ary, value)
         upper = ary.size - 1
         lower = 0
-        idx = 0
 
         while (lower <= upper) do
           idx = (lower + upper) / 2
@@ -124,7 +124,7 @@ module Dalli
             lower = idx + 1
           end
         end
-        return upper
+        upper
       end
     end
 

data/lib/dalli/server.rb

@@ -1,3 +1,4 @@
+# frozen_string_literal: true
 require 'socket'
 require 'timeout'
 
@@ -14,7 +15,7 @@ module Dalli
     DEFAULT_WEIGHT = 1
     DEFAULTS = {
       # seconds between trying to contact a remote server
-      :down_retry_delay => 1,
+      :down_retry_delay => 60,
       # connect/read/write timeout for socket operations
       :socket_timeout => 0.5,
       # times a socket operation may fail before considering the server dead
@@ -23,6 +24,8 @@ module Dalli
       :socket_failure_delay => 0.01,
       # max size of value in bytes (default is 1 MB, can be overriden with "memcached -I <size>")
       :value_max_bytes => 1024 * 1024,
+      # surpassing value_max_bytes either warns (false) or throws (true)
+      :error_when_over_max_size => false,
       :compressor => Compressor,
       # min byte size to attempt compression
       :compression_min_size => 1024,
@@ -65,16 +68,12 @@ module Dalli
       raise Dalli::NetworkError, "#{name} is down: #{@error} #{@msg}. If you are sure it is running, ensure memcached version is > 1.4." unless alive?
       begin
         send(op, *args)
-      rescue Dalli::NetworkError
-        raise
       rescue Dalli::MarshalError => ex
         Dalli.logger.error "Marshalling error for key '#{args.first}': #{ex.message}"
         Dalli.logger.error "You are trying to cache a Ruby object which cannot be serialized to memcached."
         Dalli.logger.error ex.backtrace.join("\n\t")
         false
-      rescue Dalli::DalliError
-        raise
-      rescue Timeout::Error
+      rescue Dalli::DalliError, Dalli::NetworkError, Dalli::ValueOverMaxSize, Timeout::Error
         raise
       rescue => ex
         Dalli.logger.error "Unexpected exception during Dalli request: #{ex.class.name}: #{ex.message}"
@@ -128,7 +127,7 @@ module Dalli
     def multi_response_start
       verify_state
       write_noop
-      @multi_buffer = ''
+      @multi_buffer = String.new('')
       @position = 0
       @inprogress = true
     end
@@ -274,7 +273,7 @@ module Dalli
     end
 
     def send_multiget(keys)
-      req = ""
+      req = String.new("")
       keys.each do |key|
         req << [REQUEST, OPCODES[:getkq], key.bytesize, 0, 0, 0, key.bytesize, 0, 0, key].pack(FORMAT[:getkq])
       end
@@ -448,6 +447,9 @@ module Dalli
     rescue ArgumentError
       raise if $!.message !~ /undefined class|marshal data too short/
       raise UnmarshalError, "Unable to unmarshal value: #{$!.message}"
+    rescue NameError
+      raise if $!.message !~ /uninitialized constant/
+      raise UnmarshalError, "Unable to unmarshal value: #{$!.message}"
     rescue Zlib::Error
       raise UnmarshalError, "Unable to uncompress value: #{$!.message}"
     end
@@ -475,19 +477,24 @@ module Dalli
       if value.bytesize <= @options[:value_max_bytes]
         yield
       else
-        Dalli.logger.warn "Value for #{key} over max size: #{@options[:value_max_bytes]} <= #{value.bytesize}"
+        message = "Value for #{key} over max size: #{@options[:value_max_bytes]} <= #{value.bytesize}"
+        raise Dalli::ValueOverMaxSize, message if @options[:error_when_over_max_size]
+
+        Dalli.logger.error "#{message} - this value may be truncated by memcached"
         false
       end
     end
 
-    # https://code.google.com/p/memcached/wiki/NewCommands#Standard_Protocol
+    # https://github.com/memcached/memcached/blob/master/doc/protocol.txt#L79
     # > An expiration time, in seconds. Can be up to 30 days. After 30 days, is treated as a unix timestamp of an exact date.
     MAX_ACCEPTABLE_EXPIRATION_INTERVAL = 30*24*60*60 # 30 days
     def sanitize_ttl(ttl)
       ttl_as_i = ttl.to_i
       return ttl_as_i if ttl_as_i <= MAX_ACCEPTABLE_EXPIRATION_INTERVAL
+      now = Time.now.to_i
+      return ttl_as_i if ttl_as_i > now # already a timestamp
       Dalli.logger.debug "Expiration interval (#{ttl_as_i}) too long for Memcached, converting to an expiration timestamp"
-      Time.now.to_i + ttl_as_i
+      now + ttl_as_i
     end
 
     # Implements the NullObject pattern to store an application-defined value for 'Key not found' responses.
@@ -604,7 +611,7 @@ module Dalli
     RESPONSE = 0x81
 
     # Response codes taken from:
-    # https://code.google.com/p/memcached/wiki/BinaryProtocolRevamped#Response_Status
+    # https://github.com/memcached/memcached/wiki/BinaryProtocolRevamped#response-status
     RESPONSE_CODES = {
       0 => 'No error',
       1 => 'Key not found',
@@ -698,7 +705,7 @@ module Dalli
       req = [REQUEST, OPCODES[:auth_negotiation], 0, 0, 0, 0, 0, 0, 0].pack(FORMAT[:noop])
       write(req)
 
-      (extras, type, status, count) = read_header.unpack(NORMAL_HEADER)
+      (extras, _type, status, count) = read_header.unpack(NORMAL_HEADER)
       raise Dalli::NetworkError, "Unexpected message format: #{extras} #{count}" unless extras == 0 && count > 0
       content = read(count).gsub(/\u0000/, ' ')
       return (Dalli.logger.debug("Authentication not required/supported by server")) if status == 0x81
@@ -711,7 +718,7 @@ module Dalli
       req = [REQUEST, OPCODES[:auth_request], mechanism.bytesize, 0, 0, 0, mechanism.bytesize + msg.bytesize, 0, 0, mechanism, msg].pack(FORMAT[:auth_request])
       write(req)
 
-      (extras, type, status, count) = read_header.unpack(NORMAL_HEADER)
+      (extras, _type, status, count) = read_header.unpack(NORMAL_HEADER)
       raise Dalli::NetworkError, "Unexpected message format: #{extras} #{count}" unless extras == 0 && count > 0
       content = read(count)
       return Dalli.logger.info("Dalli/SASL: #{content}") if status == 0

data/lib/dalli/socket.rb

@@ -1,3 +1,4 @@
+# frozen_string_literal: true
 require 'rbconfig'
 
 module Dalli::Server::TCPSocketOptions
@@ -27,7 +28,7 @@ begin
     alias :write :kgio_write
 
     def readfull(count)
-      value = ''
+      value = String.new('')
       while true
         value << kgio_read!(count - value.bytesize)
         break if value.bytesize == count
@@ -36,7 +37,7 @@ begin
     end
 
     def read_available
-      value = ''
+      value = String.new('')
       while true
         ret = kgio_tryread(8196)
         case ret
@@ -63,6 +64,9 @@ begin
       sock.server = server
       sock.kgio_wait_writable
       sock
+    rescue Timeout::Error
+      sock.close if sock
+      raise
     end
   end
 
@@ -74,6 +78,9 @@ begin
       sock.server = server
       sock.kgio_wait_writable
       sock
+    rescue Timeout::Error
+      sock.close if sock
+      raise
     end
   end
 
@@ -88,7 +95,7 @@ rescue LoadError
   module Dalli::Server::KSocket
     module InstanceMethods
       def readfull(count)
-        value = ''
+        value = String.new('')
         begin
           while true
             value << read_nonblock(count - value.bytesize)
@@ -98,14 +105,15 @@ rescue LoadError
           if IO.select([self], nil, nil, options[:socket_timeout])
             retry
           else
-            raise Timeout::Error, "IO timeout: #{options.inspect}"
+            safe_options = options.reject{|k,v| [:username, :password].include? k}
+            raise Timeout::Error, "IO timeout: #{safe_options.inspect}"
           end
         end
         value
       end
 
       def read_available
-        value = ''
+        value = String.new('')
         while true
           begin
             value << read_nonblock(8196)

data/lib/dalli/version.rb

@@ -1,3 +1,4 @@
+# frozen_string_literal: true
 module Dalli
-  VERSION = '2.7.6'
+  VERSION = '2.7.11'
 end

data/lib/rack/session/dalli.rb

@@ -1,3 +1,4 @@
+# frozen_string_literal: true
 require 'rack/session/abstract/id'
 require 'dalli'
 
@@ -6,46 +7,98 @@ module Rack
     class Dalli < defined?(Abstract::Persisted) ? Abstract::Persisted : Abstract::ID
       attr_reader :pool, :mutex
 
-      DEFAULT_OPTIONS = Abstract::ID::DEFAULT_OPTIONS.merge \
+      DEFAULT_DALLI_OPTIONS = {
         :namespace => 'rack:session',
         :memcache_server => 'localhost:11211'
+      }
 
+      # Brings in a new Rack::Session::Dalli middleware with the given
+      # `:memcache_server`. The server is either a hostname, or a
+      # host-with-port string in the form of "host_name:port", or an array of
+      # such strings. For example:
+      #
+      #   use Rack::Session::Dalli,
+      #     :memcache_server => "mc.example.com:1234"
+      #
+      # If no `:memcache_server` option is specified, Rack::Session::Dalli will
+      # connect to localhost, port 11211 (the default memcached port). If
+      # `:memcache_server` is set to nil, Dalli::Client will look for
+      # ENV['MEMCACHE_SERVERS'] and use that value if it is available, or fall
+      # back to the same default behavior described above.
+      #
+      # Rack::Session::Dalli is intended to be a drop-in replacement for
+      # Rack::Session::Memcache. It accepts additional options that control the
+      # behavior of Rack::Session, Dalli::Client, and an optional
+      # ConnectionPool. First and foremost, if you wish to instantiate your own
+      # Dalli::Client (or ConnectionPool) and use that instead of letting
+      # Rack::Session::Dalli instantiate it on your behalf, simply pass it in
+      # as the `:cache` option. Please note that you will be responsible for
+      # setting the namespace and any other options on Dalli::Client.
+      #
+      # Secondly, if you're not using the `:cache` option, Rack::Session::Dalli
+      # accepts the same options as Dalli::Client, so it's worth reviewing its
+      # documentation. Perhaps most importantly, if you don't specify a
+      # `:namespace` option, Rack::Session::Dalli will default to using
+      # "rack:session".
+      #
+      # Whether you are using the `:cache` option or not, it is not recommend
+      # to set `:expires_in`. Instead, use `:expire_after`, which will control
+      # both the expiration of the client cookie as well as the expiration of
+      # the corresponding entry in memcached.
+      #
+      # Rack::Session::Dalli also accepts a host of options that control how
+      # the sessions and session cookies are managed, including the
+      # aforementioned `:expire_after` option. Please see the documentation for
+      # Rack::Session::Abstract::Persisted for a detailed explanation of these
+      # options and their default values.
+      #
+      # Finally, if your web application is multithreaded, the
+      # Rack::Session::Dalli middleware can become a source of contention. You
+      # can use a connection pool of Dalli clients by passing in the
+      # `:pool_size` and/or `:pool_timeout` options. For example:
+      #
+      #   use Rack::Session::Dalli,
+      #     :memcache_server => "mc.example.com:1234",
+      #     :pool_size => 10
+      #
+      # You must include the `connection_pool` gem in your project if you wish
+      # to use pool support. Please see the documentation for ConnectionPool
+      # for more information about it and its default options (which would only
+      # be applicable if you supplied one of the two options, but not both).
+      #
       def initialize(app, options={})
+        # Parent uses DEFAULT_OPTIONS to build @default_options for Rack::Session
         super
-        @mutex = Mutex.new
-        mserv = @default_options[:memcache_server]
-        mopts = @default_options.reject{|k,v| !DEFAULT_OPTIONS.include? k }
-        @pool = options[:cache] || ::Dalli::Client.new(mserv, mopts)
-        @pool.alive!
-      end
 
-      if defined?(Abstract::Persisted)
-        def find_session(req, sid)
-          get_session req.env, sid
-        end
+        # Determine the default TTL for newly-created sessions
+        @default_ttl = ttl @default_options[:expire_after]
 
-        def write_session(req, sid, session, options)
-          set_session req.env, sid, session, options
-        end
+        # Normalize and validate passed options
+        cache, mserv, mopts, popts = extract_dalli_options options
 
-        def delete_session(req, sid, options)
-          destroy_session req.env, sid, options
-        end
-      end
+        @pool =
+          if cache # caller passed a Dalli::Client or ConnectionPool instance
+            cache
+          elsif popts # caller passed ConnectionPool options
+            ConnectionPool.new(popts) { ::Dalli::Client.new(mserv, mopts) }
+          else
+            ::Dalli::Client.new(mserv, mopts)
+          end
 
-      def generate_sid
-        while true
-          sid = super
-          break sid unless @pool.get(sid)
+        if @pool.respond_to?(:alive!) # is a Dalli::Client
+          @mutex = Mutex.new
+
+          @pool.alive!
         end
       end
 
       def get_session(env, sid)
-        with_lock(env, [nil, {}]) do
-          unless sid and !sid.empty? and session = @pool.get(sid)
-            sid, session = generate_sid, {}
-            unless @pool.add(sid, session)
-              raise "Session collision on '#{sid.inspect}'"
+        with_block(env, [nil, {}]) do |dc|
+          unless sid and !sid.empty? and session = dc.get(sid)
+            old_sid, sid, session = sid, generate_sid_with(dc), {}
+            unless dc.add(sid, session, @default_ttl)
+              sid = old_sid
+              redo # generate a new sid and try again
             end
           end
           [sid, session]
@@ -54,25 +107,66 @@ module Rack
 
       def set_session(env, session_id, new_session, options)
         return false unless session_id
-        expiry = options[:expire_after]
-        expiry = expiry.nil? ? 0 : expiry + 1
 
-        with_lock(env, false) do
-          @pool.set session_id, new_session, expiry
+        with_block(env, false) do |dc|
+          dc.set(session_id, new_session, ttl(options[:expire_after]))
           session_id
         end
       end
 
       def destroy_session(env, session_id, options)
-        with_lock(env) do
-          @pool.delete(session_id)
-          generate_sid unless options[:drop]
+        with_block(env) do |dc|
+          dc.delete(session_id)
+          generate_sid_with(dc) unless options[:drop]
+        end
+      end
+
+      if defined?(Abstract::Persisted)
+        def find_session(req, sid)
+          get_session req.env, sid
+        end
+
+        def write_session(req, sid, session, options)
+          set_session req.env, sid, session, options
+        end
+
+        def delete_session(req, sid, options)
+          destroy_session req.env, sid, options
         end
       end
 
-      def with_lock(env, default=nil)
-        @mutex.lock if env['rack.multithread']
-        yield
+      private
+
+      def extract_dalli_options(options)
+        return [options[:cache]] if options[:cache]
+
+        # Filter out Rack::Session-specific options and apply our defaults
+        mopts = DEFAULT_DALLI_OPTIONS.merge \
+          options.reject {|k, _| DEFAULT_OPTIONS.key? k }
+        mserv = mopts.delete :memcache_server
+
+        if mopts[:pool_size] || mopts[:pool_timeout]
+          popts = {}
+          popts[:size] = mopts.delete :pool_size if mopts[:pool_size]
+          popts[:timeout] = mopts.delete :pool_timeout if mopts[:pool_timeout]
+
+          # For a connection pool, locking is handled at the pool level
+          mopts[:threadsafe] = false unless mopts.key? :threadsafe
+        end
+
+        [nil, mserv, mopts, popts]
+      end
+
+      def generate_sid_with(dc)
+        while true
+          sid = generate_sid
+          break sid unless dc.get(sid)
+        end
+      end
+
+      def with_block(env, default=nil, &block)
+        @mutex.lock if @mutex and env['rack.multithread']
+        @pool.with(&block)
       rescue ::Dalli::DalliError, Errno::ECONNREFUSED
         raise if $!.message =~ /undefined class/
         if $VERBOSE
@@ -81,9 +175,12 @@ module Rack
         end
         default
       ensure
-        @mutex.unlock if @mutex.locked?
+        @mutex.unlock if @mutex and @mutex.locked?
       end
 
+      def ttl(expire_after)
+        expire_after.nil? ? 0 : expire_after + 1
+      end
     end
   end
 end

metadata

@@ -1,128 +1,16 @@
 --- !ruby/object:Gem::Specification
 name: dalli
 version: !ruby/object:Gem::Version
-  version: 2.7.6
+  version: 2.7.11
 platform: ruby
 authors:
 - Peter M. Goldstein
 - Mike Perham
-autorequire: 
+autorequire:
 bindir: bin
 cert_chain: []
-date: 2016-02-14 00:00:00.000000000 Z
-dependencies:
-- !ruby/object:Gem::Dependency
-  name: minitest
-  requirement: !ruby/object:Gem::Requirement
-    requirements:
-    - - ">="
-      - !ruby/object:Gem::Version
-        version: 4.2.0
-  type: :development
-  prerelease: false
-  version_requirements: !ruby/object:Gem::Requirement
-    requirements:
-    - - ">="
-      - !ruby/object:Gem::Version
-        version: 4.2.0
-- !ruby/object:Gem::Dependency
-  name: mocha
-  requirement: !ruby/object:Gem::Requirement
-    requirements:
-    - - ">="
-      - !ruby/object:Gem::Version
-        version: '0'
-  type: :development
-  prerelease: false
-  version_requirements: !ruby/object:Gem::Requirement
-    requirements:
-    - - ">="
-      - !ruby/object:Gem::Version
-        version: '0'
-- !ruby/object:Gem::Dependency
-  name: rails
-  requirement: !ruby/object:Gem::Requirement
-    requirements:
-    - - "~>"
-      - !ruby/object:Gem::Version
-        version: '4'
-  type: :development
-  prerelease: false
-  version_requirements: !ruby/object:Gem::Requirement
-    requirements:
-    - - "~>"
-      - !ruby/object:Gem::Version
-        version: '4'
-- !ruby/object:Gem::Dependency
-  name: rake
-  requirement: !ruby/object:Gem::Requirement
-    requirements:
-    - - ">="
-      - !ruby/object:Gem::Version
-        version: '0'
-  type: :development
-  prerelease: false
-  version_requirements: !ruby/object:Gem::Requirement
-    requirements:
-    - - ">="
-      - !ruby/object:Gem::Version
-        version: '0'
-- !ruby/object:Gem::Dependency
-  name: appraisal
-  requirement: !ruby/object:Gem::Requirement
-    requirements:
-    - - ">="
-      - !ruby/object:Gem::Version
-        version: '0'
-  type: :development
-  prerelease: false
-  version_requirements: !ruby/object:Gem::Requirement
-    requirements:
-    - - ">="
-      - !ruby/object:Gem::Version
-        version: '0'
-- !ruby/object:Gem::Dependency
-  name: connection_pool
-  requirement: !ruby/object:Gem::Requirement
-    requirements:
-    - - ">="
-      - !ruby/object:Gem::Version
-        version: '0'
-  type: :development
-  prerelease: false
-  version_requirements: !ruby/object:Gem::Requirement
-    requirements:
-    - - ">="
-      - !ruby/object:Gem::Version
-        version: '0'
-- !ruby/object:Gem::Dependency
-  name: rdoc
-  requirement: !ruby/object:Gem::Requirement
-    requirements:
-    - - ">="
-      - !ruby/object:Gem::Version
-        version: '0'
-  type: :development
-  prerelease: false
-  version_requirements: !ruby/object:Gem::Requirement
-    requirements:
-    - - ">="
-      - !ruby/object:Gem::Version
-        version: '0'
-- !ruby/object:Gem::Dependency
-  name: simplecov
-  requirement: !ruby/object:Gem::Requirement
-    requirements:
-    - - ">="
-      - !ruby/object:Gem::Version
-        version: '0'
-  type: :development
-  prerelease: false
-  version_requirements: !ruby/object:Gem::Requirement
-    requirements:
-    - - ">="
-      - !ruby/object:Gem::Version
-        version: '0'
+date: 2020-09-26 00:00:00.000000000 Z
+dependencies: []
 description: High performance memcached client for Ruby
 email:
 - peter.m.goldstein@gmail.com
@@ -152,9 +40,8 @@ homepage: https://github.com/petergoldstein/dalli
 licenses:
 - MIT
 metadata: {}
-post_install_message: 
-rdoc_options:
-- "--charset=UTF-8"
+post_install_message:
+rdoc_options: []
 require_paths:
 - lib
 required_ruby_version: !ruby/object:Gem::Requirement
@@ -168,9 +55,9 @@ required_rubygems_version: !ruby/object:Gem::Requirement
     - !ruby/object:Gem::Version
       version: '0'
 requirements: []
-rubyforge_project: 
-rubygems_version: 2.4.5.1
-signing_key: 
+rubyforge_project:
+rubygems_version: 2.7.6.2
+signing_key:
 specification_version: 4
 summary: High performance memcached client for Ruby
 test_files: []