dalli-elasticache 0.1.1 → 1.0.1

checksums.yaml

@@ -1,15 +1,7 @@
 ---
-!binary "U0hBMQ==":
-  metadata.gz: !binary |-
-    OGZlZjgwNzQyNTU3MTMzOWZkYjI4M2RiZGJlMmYwZTQ3ODBkMmMxZg==
-  data.tar.gz: !binary |-
-    YzM4Y2E3ZjlhMzA3M2NjYTc0NjZjZTU1MzA4Y2UwODM1YmZjMjNmYw==
+SHA256:
+  metadata.gz: 050e21a65939176f89e224e1cd7240051fcfec434fb8f048b8a00d17e70a175f
+  data.tar.gz: 77f1c0ff3f43bb6eff10de21005c56a36f161524408567313e40c56f94e09d85
 SHA512:
-  metadata.gz: !binary |-
-    ZjA5MmNmZjRjYTE2ZDY2NWE3ZGVkNGE0ZDYzOGY0YzI1Nzg2ODlmY2I5ZWM5
-    N2E5ZWJhZmRkYTBmYTg1Zjg3NmQ2N2Q1YjUyZTBlN2E1ZWYxMDFmMmI4MTM1
-    ZWM5ZmY0OTkzMDU2MmI4NGZiODFmY2QzNmZhMWEzY2MxZWIzZjk=
-  data.tar.gz: !binary |-
-    ZjVlMmMyMzI3ODhlMDFiMTc4NTQzMjc5ZDkyYjFlMGRmOGIwNGYxYTc5OTUw
-    YTk2YWE1MmFmMmViODZkM2UwMTliMTdhYWM2ZDRhMjljOGJkZmQ5OTc3NzAy
-    NzIyMzI5NWY1YWFmY2M5ODMwYzIzN2UzNmFhMjRiMTkwYjUwMDQ=
+  metadata.gz: 1ced0146220de8f94e8b97cbb6fcde53a5ddd3335716482b9b84feadae020d8054f4e18c63bed9dcac60b41fcca6e68e8b8ea007a2192f6985714136324c5e49
+  data.tar.gz: 0d9aced1a50c272e9974d8b1d9e4da6628409235317d46f0af5fd7db9e1b86ce3a7c07ebcbe389a80c646bf6d557dd896257af70d269ac807445f6c370b93379

data/README.md

@@ -1,46 +1,63 @@
-Dalli ElastiCache [![Gem Version](https://badge.fury.io/rb/dalli-elasticache.svg)](http://badge.fury.io/rb/dalli-elasticache) [![Build Status](https://travis-ci.org/zmillman/dalli-elasticache.svg?branch=request-refactor)](https://travis-ci.org/zmillman/dalli-elasticache) [![Code Climate](https://codeclimate.com/github/ktheory/dalli-elasticache.png)](https://codeclimate.com/github/ktheory/dalli-elasticache)
+Dalli ElastiCache [![Gem Version](https://badge.fury.io/rb/dalli-elasticache.svg)](http://badge.fury.io/rb/dalli-elasticache) [![Build Status](https://github.com/ktheory/dalli-elasticache/actions/workflows/tests.yml/badge.svg)](https://github.com/ktheory/dalli-elasticache/actions/workflows/tests.yml) [![Code Climate](https://codeclimate.com/github/ktheory/dalli-elasticache.png)](https://codeclimate.com/github/ktheory/dalli-elasticache)
 =================
 
-Use [AWS ElastiCache AutoDiscovery](http://docs.aws.amazon.com/AmazonElastiCache/latest/UserGuide/AutoDiscovery.html) to automatically configure your [Dalli memcached client](https://github.com/mperham/dalli) with all the nodes in your cluster.
+Use [AWS ElastiCache AutoDiscovery](http://docs.aws.amazon.com/AmazonElastiCache/latest/UserGuide/AutoDiscovery.html) or [Google Cloud MemoryStore Auto Discovery](https://cloud.google.com/memorystore/docs/memcached/using-auto-discovery) to automatically configure your [Dalli memcached client](https://github.com/petergoldstein/dalli) with all the nodes in your cluster.
 
 Installation
 ------------
 
-Install the [rubygem](https://rubygems.org/gems/dalli-elasticache):
+Install the [gem](https://rubygems.org/gems/dalli-elasticache):
 
 ```ruby
 # in your Gemfile
 gem 'dalli-elasticache'
 ```
 
-Setup for Rails 3.x and Newer
------------------------------
+Using Dalli Elasticache in Rails
+---------------------------------
 
-Configure your environment-specific application settings:
+Note that the list of memcached servers used by Rails will be refreshed each time an app server process starts.  If the list of nodes in your cluster changes, this configuration will not be reflected in the Rails configuraiton without such a server process restart.
+
+### Configuring a Cache Store
+
+The most common use of Dalli in Rails is to support a cache store.  To set up your cache store with a cluster, you'll need to generate the list of servers with Dalli ElastiCache and pass them to the `cache_store` configuration.  This needs to be done in your `config/environments/RAILS_ENV.rb` file for each Rails environment where you want to use a cluster.
 
 ```ruby
 # in config/environments/production.rb
-config_endpoint = "my-cluster-name.abc123.cfg.use1.cache.amazonaws.com:1211"
-elasticache     = Dalli::ElastiCache.new(endpoint)
+endpoint    = "my-cluster-name.abc123.cfg.use1.cache.amazonaws.com:11211"
+elasticache = Dalli::ElastiCache.new(endpoint)
 
-config.cache_store = :dalli_store, elasticache.servers, {:expires_in => 1.day, :compress => true}
+config.cache_store = :mem_cache_store, elasticache.servers, { expires_in: 1.day }
 ```
+### Configuring a Session Store
+
+Another use of Dalli in Rails is to support a Rails session store.  Dalli ElastiCache can also be used in this case.  The usage is very similar - first use Dalli ElastiCache to generate the list of servers, and then pass that result to the Rails configuration.  In `config/application.rb` you would write:
 
-Note that the ElastiCache server list will be refreshed each time an app server process starts.
+```ruby
+# in config/environments/production.rb
+endpoint    = "my-cluster-name.abc123.cfg.use1.cache.amazonaws.com:11211"
+elasticache = Dalli::ElastiCache.new(endpoint)
 
-Client Usage
+config.session_store = :mem_cache_store, memcache_server: elasticache.servers, pool_size: 10, pool_timeout: 5, expire_after: 1.day
+```
+
+### Dalli Considerations
+
+Please see [here](https://github.com/petergoldstein/dalli/wiki/Using-Dalli-with-Rails) for more information on configuring Dalli and Rails.
+
+
+Using Dalli ElastiCache with a Dalli Client
 ------------
 
-Create an ElastiCache instance:
+To initialize a Dalli Client the is configured for all the nodes of a cluster, one simply needs to pass the configuration endpoint and any options for the Dalli Client into the `Dalli::ElastiCache` initializer.  Then one can use the methods on the `Dalli::ElastiCache` object to generate an appropriately configured `Dalli::Client`or to get information about the cluster.
 
 ```ruby
 config_endpoint = "aaron-scratch.vfdnac.cfg.use1.cache.amazonaws.com:11211"
 
 # Options for configuring the Dalli::Client
 dalli_options = {
-  :expires_in => 24 * 60 * 60,
-  :namespace => "my_app",
-  :compress => true
+  expires_in: 24 * 60 * 60,
+  namespace: "my_app"
 }
 
 elasticache = Dalli::ElastiCache.new(config_endpoint, dalli_options)
@@ -51,11 +68,11 @@ Fetch information about the Memcached nodes:
 ```ruby
 # Dalli::Client with configuration from the AutoDiscovery endpoint
 elasticache.client
-# => #<Dalli::Client ... @servers=["10.84.227.155:11211", ...]>
+# => #<Dalli::Client ... @servers=["aaron-scratch.vfdnac.0001.use1.cache.amazonaws.com:11211", ...]>
 
-# Node IP addresses and hostnames
+# Node addresses
 elasticache.servers
-# => ["10.84.227.115:11211", "10.77.71.127:11211"]
+# => ["aaron-scratch.vfdnac.0001.use1.cache.amazonaws.com:11211", "aaron-scratch.vfdnac.0002.use1.cache.amazonaws.com:11211"]
 
 # Number of times the cluster configuration has changed
 elasticache.version
@@ -75,6 +92,4 @@ elasticache.refresh.client
 License
 -------
 
-Copyright 2013 Aaron Suggs
-
-Released under an [MIT License](http://opensource.org/licenses/MIT)
+Copyright (2017-2022) Aaron Suggs, Peter M. Goldstein. See LICENSE for details.

data/Rakefile

@@ -1,7 +1,10 @@
-require "rubygems"
-require "bundler/setup"
-require "rspec/core/rake_task"
+# frozen_string_literal: true
+
+require 'rubygems'
+require 'bundler/setup'
+require 'bundler/gem_tasks'
+require 'rspec/core/rake_task'
 
 RSpec::Core::RakeTask.new(:test)
 
-task :default => :test
+task default: :test

data/lib/dalli/elasticache/auto_discovery/base_command.rb

@@ -0,0 +1,42 @@
+# frozen_string_literal: true
+
+module Dalli
+  module Elasticache
+    module AutoDiscovery
+      ##
+      # Base command class for configuration endpoint
+      # command.  Contains the network logic.
+      ##
+      class BaseCommand
+        attr_reader :host, :port
+
+        def initialize(host, port)
+          @host = host
+          @port = port
+        end
+
+        # Send an ASCII command to the endpoint
+        #
+        # Returns the raw response as a String
+        def send_command
+          socket = TCPSocket.new(@host, @port)
+          begin
+            socket.puts command
+            response_from_socket(socket)
+          ensure
+            socket.close
+          end
+        end
+
+        def response_from_socket(socket)
+          data = +''
+          until (line = socket.readline).include?('END')
+            data << line
+          end
+
+          data
+        end
+      end
+    end
+  end
+end

data/lib/dalli/elasticache/auto_discovery/config_command.rb

@@ -0,0 +1,45 @@
+# frozen_string_literal: true
+
+module Dalli
+  module Elasticache
+    module AutoDiscovery
+      ##
+      # Encapsulates execution of the 'config' command, which is used to
+      # extract the list of nodes and determine if that list of nodes has changed.
+      ##
+      class ConfigCommand < BaseCommand
+        attr_reader :engine_version
+
+        CONFIG_COMMAND = "config get cluster\r\n"
+
+        # Legacy command for version < 1.4.14
+        LEGACY_CONFIG_COMMAND = "get AmazonElastiCache:cluster\r\n"
+
+        def initialize(host, port, engine_version)
+          super(host, port)
+          @engine_version = engine_version
+        end
+
+        def response
+          ConfigResponse.new(send_command)
+        end
+
+        def command
+          return LEGACY_CONFIG_COMMAND if legacy_config?
+
+          CONFIG_COMMAND
+        end
+
+        def legacy_config?
+          return false unless engine_version
+          return false if engine_version.casecmp('unknown').zero?
+
+          Gem::Version.new(engine_version) < Gem::Version.new('1.4.14')
+        rescue ArgumentError
+          # Just assume false if we can't parse the engine_version
+          false
+        end
+      end
+    end
+  end
+end

data/lib/dalli/elasticache/auto_discovery/config_response.rb

@@ -1,48 +1,45 @@
+# frozen_string_literal: true
+
 module Dalli
   module Elasticache
     module AutoDiscovery
-      
       # This class wraps the raw ASCII response from an Auto Discovery endpoint
       # and provides methods for extracting data from that response.
       #
       # http://docs.aws.amazon.com/AmazonElastiCache/latest/UserGuide/AutoDiscovery.AddingToYourClientLibrary.html
-      
       class ConfigResponse
-        
         # The raw response text
         attr_reader :text
-    
+
         # Matches the version line of the response
-        VERSION_REGEX = /^(\d+)$/
-    
+        VERSION_REGEX = /^(\d+)\r?\n/.freeze
+
         # Matches strings like "my-cluster.001.cache.aws.com|10.154.182.29|11211"
-        NODE_REGEX = /(([-.a-zA-Z0-9]+)\|(\b\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}\b)\|(\d+))/
-        NODE_LIST_REGEX = /^(#{NODE_REGEX}\s*)+$/
-    
+        NODE_REGEX = /(([-.a-zA-Z0-9]+)\|(\b\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}\b)\|(\d+))/.freeze
+        NODE_LIST_REGEX = /^(#{NODE_REGEX}\s*)+$/.freeze
+
         def initialize(response_text)
           @text = response_text.to_s
         end
-    
+
         # The number of times the configuration has been changed
         #
         # Returns an integer
         def version
-          VERSION_REGEX.match(@text)[1].to_i
+          m = VERSION_REGEX.match(@text)
+          return -1 unless m
+
+          m[1].to_i
         end
-    
+
         # Node hosts, ip addresses, and ports
         #
         # Returns an Array of Hashes with values for :host, :ip and :port
         def nodes
           NODE_LIST_REGEX.match(@text).to_s.scan(NODE_REGEX).map do |match|
-            {
-              :host => match[1],
-              :ip   => match[2],
-              :port => match[3].to_i
-            }
+            Node.new(match[1], match[2], match[3].to_i)
           end
         end
-    
       end
     end
   end

data/lib/dalli/elasticache/auto_discovery/endpoint.rb

@@ -1,76 +1,45 @@
+# frozen_string_literal: true
+
 module Dalli
   module Elasticache
     module AutoDiscovery
+      ##
+      # This is a representation of the configuration endpoint for
+      # a memcached cluster.  It encapsulates information returned from
+      # that endpoint.
+      ##
       class Endpoint
-        
         # Endpoint configuration
-        attr_reader :host
-        attr_reader :port
-    
+        attr_reader :host, :port
+
         # Matches Strings like "my-host.cache.aws.com:11211"
-        ENDPOINT_REGEX = /([-.a-zA-Z0-9]+):(\d+)/
-    
-        STATS_COMMAND  = "stats\r\n"
-        CONFIG_COMMAND = "config get cluster\r\n"
-        # Legacy command for version < 1.4.14
-        OLD_CONFIG_COMMAND = "get AmazonElastiCache:cluster\r\n"
-    
-        def initialize(endpoint)
-          ENDPOINT_REGEX.match(endpoint) do |m|
-            @host = m[1]
-            @port = m[2].to_i
-          end
-        end
-    
+        ENDPOINT_REGEX = /^([-_.a-zA-Z0-9]+)(?::(\d+))?$/.freeze
+
+        def initialize(addr)
+          @host, @port = parse_endpoint_address(addr)
+        end
+
+        DEFAULT_PORT = 11_211
+        def parse_endpoint_address(addr)
+          m = ENDPOINT_REGEX.match(addr)
+          raise ArgumentError, "Unable to parse configuration endpoint address - #{addr}" unless m
+
+          [m[1], (m[2] || DEFAULT_PORT).to_i]
+        end
+
         # A cached ElastiCache::StatsResponse
         def stats
-          @stats ||= get_stats_from_remote
+          @stats ||= StatsCommand.new(@host, @port).response
         end
-    
+
         # A cached ElastiCache::ConfigResponse
         def config
-          @config ||= get_config_from_remote
+          @config ||= ConfigCommand.new(@host, @port, engine_version).response
         end
-    
+
         # The memcached engine version
         def engine_version
-          stats.version
-        end
-    
-        protected
-    
-        def with_socket(&block)
-          TCPSocket.new(config_host, config_port)
-        end
-    
-        def get_stats_from_remote
-          data = remote_command(STATS_COMMAND)
-          StatsResponse.new(data)
-        end
-    
-        def get_config_from_remote
-          if engine_version < Gem::Version.new("1.4.14")
-            data = remote_command(OLD_CONFIG_COMMAND)
-          else
-            data = remote_command(CONFIG_COMMAND)
-          end
-          ConfigResponse.new(data)
-        end
-      
-        # Send an ASCII command to the endpoint
-        #
-        # Returns the raw response as a String
-        def remote_command(command)
-          socket = TCPSocket.new(@host, @port)
-          socket.puts command
-        
-          data = ""
-          until (line = socket.readline) =~ /END/
-            data << line
-          end
-        
-          socket.close
-          data
+          stats.engine_version
         end
       end
     end

data/lib/dalli/elasticache/auto_discovery/node.rb

@@ -0,0 +1,39 @@
+# frozen_string_literal: true
+
+module Dalli
+  module Elasticache
+    module AutoDiscovery
+      ##
+      # Represents a single memcached node in the
+      # cluster.
+      ##
+      class Node
+        attr_reader :host, :ip, :port
+
+        def initialize(host, ip, port)
+          @host = host
+          @ip = ip
+          @port = port
+        end
+
+        def ==(other)
+          host == other.host &&
+            ip == other.ip &&
+            port == other.port
+        end
+
+        def eql?(other)
+          self == other
+        end
+
+        def hash
+          [host, ip, port].hash
+        end
+
+        def to_s
+          "#{@host}:#{@port}"
+        end
+      end
+    end
+  end
+end

data/lib/dalli/elasticache/auto_discovery/stats_command.rb

@@ -0,0 +1,23 @@
+# frozen_string_literal: true
+
+module Dalli
+  module Elasticache
+    module AutoDiscovery
+      ##
+      # Encapsulates execution of the 'stats' command, which is used to
+      # extract the engine_version
+      ##
+      class StatsCommand < BaseCommand
+        STATS_COMMAND = "stats\r\n"
+
+        def response
+          StatsResponse.new(send_command)
+        end
+
+        def command
+          STATS_COMMAND
+        end
+      end
+    end
+  end
+end

data/lib/dalli/elasticache/auto_discovery/stats_response.rb

@@ -1,29 +1,32 @@
+# frozen_string_literal: true
+
 module Dalli
   module Elasticache
     module AutoDiscovery
-      
-      # This class wraps the raw ASCII response from an Auto Discovery endpoint
-      # and provides methods for extracting data from that response.
+      # This class wraps the raw ASCII response from a stats call to an
+      # Auto Discovery endpoint and provides methods for extracting data
+      # from that response.
       #
       # http://docs.aws.amazon.com/AmazonElastiCache/latest/UserGuide/AutoDiscovery.AddingToYourClientLibrary.html
-      
       class StatsResponse
-        
         # The raw response text
         attr_reader :text
-        
+
         # Matches the version line of the response
-        VERSION_REGEX = /^STAT version ([0-9.]+)\s*$/
-        
+        VERSION_REGEX = /^STAT version ([0-9.]+|unknown)\s*/i.freeze
+
         def initialize(response_text)
           @text = response_text.to_s
         end
-        
+
         # Extract the engine version stat
         #
-        # Returns a Gem::Version
-        def version
-          Gem::Version.new(VERSION_REGEX.match(@text)[1])
+        # Returns a string
+        def engine_version
+          m = VERSION_REGEX.match(@text)
+          return '' unless m && m[1]
+
+          m[1]
         end
       end
     end

data/lib/dalli/elasticache/version.rb

@@ -1,5 +1,7 @@
+# frozen_string_literal: true
+
 module Dalli
   class ElastiCache
-    VERSION = "0.1.1"
+    VERSION = '1.0.1'
   end
 end

data/lib/dalli/elasticache.rb

@@ -1,45 +1,73 @@
+# frozen_string_literal: true
+
 require 'dalli'
 require 'socket'
-require 'dalli/elasticache/version'
-require 'dalli/elasticache/auto_discovery/endpoint'
-require 'dalli/elasticache/auto_discovery/config_response'
-require 'dalli/elasticache/auto_discovery/stats_response'
+require_relative 'elasticache/version'
+require_relative 'elasticache/auto_discovery/endpoint'
+require_relative 'elasticache/auto_discovery/base_command'
+require_relative 'elasticache/auto_discovery/node'
+require_relative 'elasticache/auto_discovery/config_response'
+require_relative 'elasticache/auto_discovery/config_command'
+require_relative 'elasticache/auto_discovery/stats_response'
+require_relative 'elasticache/auto_discovery/stats_command'
 
 module Dalli
+  ##
+  # Dalli::Elasticache provides an interface for providing a configuration
+  # endpoint for a memcached cluster on ElasticCache and retrieving the
+  # list of addresses (hostname and port) for the individual nodes of that cluster.
+  #
+  # This allows the caller to pass that server list to Dalli, which then
+  # distributes cached items consistently over the nodes.
+  ##
   class ElastiCache
     attr_reader :endpoint, :options
-    
-    def initialize(config_endpoint, options={})
+
+    ##
+    # Creates a new Dalli::ElasticCache instance.
+    #
+    # config_endpoint - a String containing the host and (optionally) port of the
+    # configuration endpoint for the cluster.  If not specified the port will
+    # default to 11211.  The host must be either a DNS name or an IPv4 address.  IPv6
+    # addresses are not handled at this time.
+    # dalli_options - a set of options passed to the Dalli::Client that is returned
+    # by the client method.  Otherwise unused.
+    ##
+    def initialize(config_endpoint, dalli_options = {})
       @endpoint = Dalli::Elasticache::AutoDiscovery::Endpoint.new(config_endpoint)
-      @options = options
+      @options = dalli_options
     end
-    
+
     # Dalli::Client configured to connect to the cluster's nodes
     def client
       Dalli::Client.new(servers, options)
     end
-    
+
     # The number of times the cluster configuration has been changed
     #
     # Returns an integer
     def version
       endpoint.config.version
     end
-    
+
     # The cache engine version of the cluster
+    #
+    # Returns a string
     def engine_version
       endpoint.engine_version
     end
-    
+
     # List of cluster server nodes with ip addresses and ports
+    # Always use host name instead of private elasticache IPs as internal IPs can change after a node is rebooted
     def servers
-      endpoint.config.nodes.map{ |h| "#{h[:ip]}:#{h[:port]}" }
+      endpoint.config.nodes.map(&:to_s)
     end
-    
+
     # Clear all cached data from the cluster endpoint
     def refresh
+      config_endpoint = "#{endpoint.host}:#{endpoint.port}"
       @endpoint = Dalli::Elasticache::AutoDiscovery::Endpoint.new(config_endpoint)
-      
+
       self
     end
   end

data/lib/dalli-elasticache.rb

@@ -1,2 +1,4 @@
+# frozen_string_literal: true
+
 # Support default bundler require path
-require 'dalli/elasticache'
+require_relative 'dalli/elasticache'