dalli 0.9.0 → 0.9.1

data/Gemfile

@@ -1,3 +1,7 @@
 source :rubygems
 
-gemspec
+gemspec
+
+group :test do
+  gem 'simplecov-html'
+end

data/History.md

@@ -0,0 +1,14 @@
+Dalli Changelog
+=====================
+
+0.9.1
+----
+
+ - Add fetch and cas operations (mperham)
+ - Add incr and decr operations (mperham)
+ - Initial support for SASL authentication via the MEMCACHE_{USERNAME,PASSWORD} environment variables, needed for Heroku (mperham)
+
+0.9.0
+-----
+
+ - Initial gem release.

data/Performance.md

@@ -2,23 +2,24 @@ Performance
 ====================
 
 Caching is all about performance, so I carefully track Dalli performance to ensure no regressions.
+Times are from a Unibody MBP 2.4Ghz Core 2 Duo running Snow Leopard.
 
-	Testing 1.8.5 with ruby 1.9.2p0 (2010-08-18 revision 29036) [x86_64-darwin10.4.0]
-	                                     user     system      total        real
-	set:plain:memcache-client        1.600000   0.390000   1.990000 (  2.020491)
-	set:ruby:memcache-client         1.680000   0.390000   2.070000 (  2.108217)
-	get:plain:memcache-client        1.740000   0.250000   1.990000 (  2.018315)
-	get:ruby:memcache-client         1.790000   0.250000   2.040000 (  2.065529)
-	multiget:ruby:memcache-client    0.800000   0.090000   0.890000 (  0.914336)
-	missing:ruby:memcache-client     1.480000   0.250000   1.730000 (  1.761555)
-	mixed:ruby:memcache-client       3.470000   0.640000   4.110000 (  4.195236)
+    Testing 1.8.5 with ruby 1.9.2p0 (2010-08-18 revision 29036) [x86_64-darwin10.4.0]
+                                         user     system      total        real
+    set:plain:memcache-client        2.070000   0.590000   2.660000 (  2.669744)
+    set:ruby:memcache-client         2.150000   0.570000   2.720000 (  2.734616)
+    get:plain:memcache-client        2.240000   0.400000   2.640000 (  2.675747)
+    get:ruby:memcache-client         2.290000   0.380000   2.670000 (  2.682108)
+    multiget:ruby:memcache-client    1.030000   0.140000   1.170000 (  1.174503)
+    missing:ruby:memcache-client     1.900000   0.370000   2.270000 (  2.282264)
+    mixed:ruby:memcache-client       4.430000   0.950000   5.380000 (  5.420251)
 
-	Testing 0.1.0 with ruby 1.9.2p0 (2010-08-18 revision 29036) [x86_64-darwin10.4.0]
-	                                     user     system      total        real
-	set:plain:dalli                  0.430000   0.180000   0.610000 (  1.051395)
-	set:ruby:dalli                   0.490000   0.180000   0.670000 (  1.124848)
-	get:plain:dalli                  0.490000   0.210000   0.700000 (  1.141887)
-	get:ruby:dalli                   0.540000   0.200000   0.740000 (  1.188353)
-	multiget:ruby:dalli              0.510000   0.200000   0.710000 (  0.772860)
-	missing:ruby:dalli               0.450000   0.210000   0.660000 (  1.070748)
-	mixed:ruby:dalli                 1.050000   0.390000   1.440000 (  2.304933)
+    Testing 0.9.0 with ruby 1.9.2p0 (2010-08-18 revision 29036) [x86_64-darwin10.4.0]
+                                         user     system      total        real
+    set:plain:dalli                  1.610000   0.360000   1.970000 (  2.032947)
+    set:ruby:dalli                   1.690000   0.360000   2.050000 (  2.108120)
+    get:plain:dalli                  1.710000   0.400000   2.110000 (  2.123895)
+    get:ruby:dalli                   1.760000   0.390000   2.150000 (  2.170964)
+    multiget:ruby:dalli              0.950000   0.310000   1.260000 (  1.269679)
+    missing:ruby:dalli               1.650000   0.380000   2.030000 (  2.054383)
+    mixed:ruby:dalli                 3.470000   0.750000   4.220000 (  4.323265)

data/README.md

@@ -1,10 +1,12 @@
 Dalli
 =========
 
-Dalli is a high performance pure Ruby client for accessing memcached servers.  It works with memcached 1.4+ only as it uses the newer binary protocol.  The API tries to be mostly compatible with memcache-client with the goal being to make it a drop-in replacement for Rails.
+Dalli is a high performance pure Ruby client for accessing memcached servers.  It works with memcached 1.4+ only as it uses the newer binary protocol.  It should be considered a replacement for the memcache-client gem.  The API tries to be mostly compatible with memcache-client with the goal being to make it a drop-in replacement for Rails.
 
 The name is a variant of Salvador Dali for his famous painting [The Persistence of Memory](http://en.wikipedia.org/wiki/The_Persistence_of_Memory).
 
+![Persistence of Memory](http://www.virtualdali.com/assets/paintings/31PersistenceOfMemory.jpg)
+
 Dalli's development is sponsored by [NorthScale](http://www.northscale.com/).  Many thanks to them!
 
 
@@ -24,11 +26,14 @@ So a few notes.  Dalli:
  1. is approximately 20% faster than memcache-client (which itself was heavily optimized) in Ruby 1.9.2.
  2. contains explicit "chokepoint" methods which handle all requests; these can be hooked into by monitoring tools (NewRelic, Rack::Bug, etc) to track memcached usage.
  3. comes with hooks to replace memcache-client in Rails.
+ 4. is approx 700 lines of Ruby.  memcache-client is approx 1250 lines.
 
 
 Installation and Usage
 ------------------------
 
+Remember, Dalli **requires** memcached 1.4+.  You can check the version with `memcached -h`.
+
     gem install dalli
 
     require 'dalli'
@@ -36,6 +41,8 @@ Installation and Usage
     dc.set('abc', 123)
     value = dc.get('abc')
 
+The test suite requires memcached 1.4.3+ with SASL enabled (./configure --enable-sasl).  Currently only supports the PLAIN mechanism.
+
 
 Usage with Rails
 ---------------------------
@@ -63,6 +70,12 @@ By default, Dalli is thread-safe.  Disable thread-safety at your own peril.
 Note that Dalli does not require ActiveSupport or Rails.  You can safely use it in your own Ruby projects.
 
 
+Helping Out
+-------------
+
+If you have a fix you wish to provide, please fork the code, fix in your local project and then send a pull request on github.  Please ensure that you include a test which verifies your fix and update History.md with a one sentence description of your fix so you get credit as a contributor.
+
+
 Author
 ----------
 

data/TODO.md

@@ -0,0 +1,6 @@
+TODO
+========
+
+ * More of the memcached instruction set.  This will be done based on user demand.  Email me if the API is missing a feature you'd like to use.
+ * Better API documentation
+ * SASL for Heroku memcached add-on support.

data/dalli.gemspec

@@ -7,12 +7,14 @@ Gem::Specification.new do |s|
   s.version = Dalli::VERSION
 
   s.authors = ["Mike Perham"]
-  s.date = %q{2010-08-19}
+  s.date = %q{2010-08-28}
   s.description = %q{High performance memcached client for Ruby}
   s.email = %q{mperham@gmail.com}
   s.files = Dir.glob("lib/**/*") + [
      "LICENSE",
      "README.md",
+     "History.md",
+     "TODO.md",
      "Rakefile",
      "Gemfile",
      "dalli.gemspec",
@@ -24,7 +26,8 @@ Gem::Specification.new do |s|
   s.summary = %q{High performance memcached client for Ruby}
   s.test_files = Dir.glob("test/**/*")
   s.add_development_dependency(%q<shoulda>, [">= 0"])
-  s.add_development_dependency(%q<rails>, [">= 3.0.0.rc"])
+  s.add_development_dependency(%q<mocha>, [">= 0"])
+  s.add_development_dependency(%q<rails>, [">= 3.0.0.rc2"])
   s.add_development_dependency(%q<memcache-client>, [">= 1.8.5"])
 end
 

data/lib/dalli.rb

@@ -4,21 +4,27 @@ require 'dalli/server'
 require 'dalli/version'
 require 'dalli/options'
 
-require 'logger'
-
 module Dalli
-  # socket communication error
+  # generic error
   class DalliError < RuntimeError; end
+  # socket/server communication error
   class NetworkError < DalliError; end
 
   def self.logger
     @logger ||= begin
       (defined?(Rails) && Rails.logger) ||
       (defined?(RAILS_DEFAULT_LOGGER) && RAILS_DEFAULT_LOGGER) ||
-      (l = Logger.new(STDOUT); l.level = Logger::INFO; l)
+      default_logger
     end
   end
 
+  def self.default_logger
+    require 'logger'
+    l = Logger.new(STDOUT)
+    l.level = Logger::INFO
+    l
+  end
+
   def self.logger=(logger)
     @logger = logger
   end

data/lib/dalli/client.rb

@@ -12,19 +12,19 @@ module Dalli
     # Both weight and port are optional.
     #
     # Options:
+    #   :failover - if a server is down, store the value on another server.  Default: true.
     #   :threadsafe - ensure that only one thread is actively using a socket at a time. Default: true.
     #   :marshal - ensure that the value you store is exactly what is returned.  Otherwise you can see this:
     #        set('abc', 123)
     #        get('abc') ==> '123'  (Note you set an Integer but got back a String)
     #      Default: true.
     #
-    def initialize(servers, options={})
+    def initialize(servers=nil, options={})
       @ring = Dalli::Ring.new(
-        Array(servers).map do |s| 
+        Array(servers || env_servers).map do |s| 
           Dalli::Server.new(s)
-        end
+        end, options
       )
-      @ring.threadsafe! unless options[:threadsafe] == false
       self.extend(Dalli::Marshal) unless options[:marshal] == false
     end
     
@@ -46,13 +46,31 @@ module Dalli
         values.inject(values) { |memo, (k,v)| memo[k] = deserialize(v); memo }
       end
     end
-    
+
+    def fetch(key, ttl=0)
+      val = get(key)
+      if val.nil? && block_given?
+        val = yield
+        add(key, val, ttl)
+      end
+      val
+    end
+
+    def cas(key, ttl=0, &block)
+      (value, cas) = perform(:cas, key)
+      value = (!value || value == 'Not found') ? nil : deserialize(value)
+      if value
+        newvalue = block.call(value)
+        perform(:add, key, serialize(newvalue), ttl, cas)
+      end
+    end
+
     def set(key, value, ttl=0)
       perform(:set, key, serialize(value), ttl)
     end
     
     def add(key, value, ttl=0)
-      perform(:add, key, serialize(value), ttl)
+      perform(:add, key, serialize(value), ttl, 0)
     end
 
     def replace(key, value, ttl=0)
@@ -64,11 +82,11 @@ module Dalli
     end
 
     def append(key, value)
-      perform(:append, key, value)
+      perform(:append, key, value.to_s)
     end
 
     def prepend(key, value)
-      perform(:prepend, key, value)
+      perform(:prepend, key, value.to_s)
     end
 
     def flush(delay=0)
@@ -80,12 +98,34 @@ module Dalli
       flush(0)
     end
 
-    def incr(key, amt)
-      perform(:incr, key, amt)
+    ##
+    # Incr adds the given amount to the counter on the memcached server.
+    # Amt must be a positive value.
+    # 
+    # memcached counters are unsigned and cannot hold negative values.  Calling
+    # decr on a counter which is 0 will just return 0.
+    #
+    # If default is nil, the counter must already exist or the operation
+    # will fail and will return nil.  Otherwise this method will return
+    # the new value for the counter.
+    def incr(key, amt, ttl=0, default=nil)
+      raise ArgumentError, "Positive values only: #{amt}" if amt < 0
+      perform(:incr, key, amt, ttl, default)
     end
 
-    def decr(key, amt)
-      perform(:decr, key, amt)
+    ##
+    # Decr subtracts the given amount from the counter on the memcached server.
+    # Amt must be a positive value.
+    # 
+    # memcached counters are unsigned and cannot hold negative values.  Calling
+    # decr on a counter which is 0 will just return 0.
+    #
+    # If default is nil, the counter must already exist or the operation
+    # will fail and will return nil.  Otherwise this method will return
+    # the new value for the counter.
+    def decr(key, amt, ttl=0, default=nil)
+      raise ArgumentError, "Positive values only: #{amt}" if amt < 0
+      perform(:decr, key, amt, ttl, default)
     end
 
     def stats
@@ -94,6 +134,7 @@ module Dalli
 
     def close
       @ring.servers.map { |s| s.close }
+      @ring = nil
     end
 
     private
@@ -106,6 +147,11 @@ module Dalli
       value
     end
 
+    def env_servers
+      ENV['MEMCACHE_SERVERS'].split(',')
+    end
+
+    # Chokepoint method for instrumentation
     def perform(op, *args)
       key = args.first
       validate_key(key)

data/lib/dalli/ring.rb

@@ -7,7 +7,7 @@ module Dalli
     
     attr_accessor :servers, :continuum
     
-    def initialize(servers)
+    def initialize(servers, options)
       @servers = servers
       if servers.size > 1
         total_weight = servers.inject(0) { |memo, srv| memo + srv.weight }
@@ -22,6 +22,9 @@ module Dalli
         continuum.sort { |a, b| a.value <=> b.value }
         @continuum = continuum
       end
+
+      threadsafe! unless options[:threadsafe] == false
+      @failover = options[:failover] != false
     end
     
     def server_for_key(key)
@@ -33,18 +36,13 @@ module Dalli
         entryidx = self.class.binary_search(@continuum, hkey)
         server = @continuum[entryidx].server
         return server if server.alive?
+        break unless @failover
         hkey = hash_for("#{try}#{key}")
       end
 
       raise Dalli::NetworkError, "No servers available"
     end
     
-    def threadsafe!
-      @servers.each do |s|
-        s.extend(Dalli::Threadsafe)
-      end
-    end
-    
     def lock
       @servers.each { |s| s.lock! }
       begin
@@ -55,7 +53,13 @@ module Dalli
     end
     
     private
-    
+
+    def threadsafe!
+      @servers.each do |s|
+        s.extend(Dalli::Threadsafe)
+      end
+    end
+
     def hash_for(key)
       Zlib.crc32(key)
     end

data/lib/dalli/sasl/anonymous.rb

@@ -0,0 +1,14 @@
+module SASL
+  ##
+  # SASL ANONYMOUS where you only send a username that may not get
+  # evaluated by the server.
+  #
+  # RFC 4505:
+  # http://tools.ietf.org/html/rfc4505
+  class Anonymous < Mechanism
+    def start
+      @state = nil
+      ['auth', preferences.username.to_s]
+    end
+  end
+end

data/lib/dalli/sasl/base.rb

@@ -0,0 +1,89 @@
+module SASL
+  
+  MECHANISMS = {
+  }
+  
+  class Preferences
+    def authzid
+      nil
+    end
+
+    def realm
+      raise NotImplementedError
+    end
+
+    def digest_uri
+      raise NotImplementedError
+    end
+
+    def username
+      ENV['MEMCACHE_USERNAME']
+    end
+
+    def has_password?
+      false
+    end
+
+    def allow_plaintext?
+      false
+    end
+
+    def password
+      ENV['MEMCACHE_PASSWORD']
+    end
+
+    def want_anonymous?
+      false
+    end
+  end
+
+  def SASL.new(mechanisms)
+    mechanisms.each do |mech|
+      if MECHANISMS.has_key?(mech)
+        x = MECHANISMS[mech]
+        return x.new(mech, Preferences.new)
+      end
+    end
+
+    raise NotImplementedError, "No supported mechanisms in #{mechanisms.join(',')}"
+  end
+
+  ##
+  # Common functions for mechanisms
+  #
+  # Mechanisms implement handling of methods start and receive. They
+  # return: [message_name, content] or nil where message_name is
+  # either 'auth' or 'response' and content is either a string which
+  # may transmitted encoded as Base64 or nil.
+  class Mechanism
+    attr_reader :preferences
+    attr_reader :name
+
+    def initialize(name, preferences)
+      @name = name
+      @preferences = preferences
+      @state = nil
+    end
+
+    def success?
+      @state == :success
+    end
+    def failure?
+      @state == :failure
+    end
+
+    def start
+      raise NotImplementedError
+    end
+
+    def receive(message_name, content)
+      case message_name
+      when 'success'
+        @state = :success
+      when 'failure'
+        @state = :failure
+      end
+      nil
+    end
+  end
+end