ngmoco-cache-money 0.2.23 → 0.2.24.2

data/README

@@ -117,9 +117,7 @@ Nested transactions are fully supported, with partial rollback and (apparent) pa
 
 ### Mocks ###
 
-For your unit tests, it is faster to use a Memcached mock than the real deal. Just place this in your initializer for your test environment:
-
-    $memcache = Cash::Mock.new
+For your unit tests, it is faster to use a Memcached mock than the real deal. In your test environment, initialize the repository with an instance of Cash::Mock.
     
 ### Locks ###
 
@@ -133,39 +131,41 @@ In most cases locks are unnecessary; the transactional Memcached client will tak
 
 Sometimes your code will request the same cache key twice in one request. You can avoid a round trip to the Memcached server by using a local, per-request cache. Add this to your initializer:
 
-  $local = Cash::Local.new($memcache)
-  $cache = Cash::Transactional.new($local, $lock)
-
+    $memcache = MemcachedWrapper.new(config[:servers].gsub(' ', '').split(','), config)
+    $local = Cash::Local.new($memcache)
+    $lock  = Cash::Lock.new($memcache)
+    $cache = Cash::Transactional.new($local, $lock)
+    
 ## Installation ##
 
-#### Step 0: Install MemCached
-
 #### Step 1: Get the GEM ####
 
-    % gem sources -a http://gems.github.com
     % sudo gem install ngmoco-cache-money
     
-#### Step 2: Configure MemCached.
+    Add the gem you your Gemfile:
+    gem 'ngmoco-cache-money', :lib => 'cache_money'
+    
+#### Step 2: Configure cache client
+
+In your environment, create a cache client instance configured for your cache servers.
+  
+    $memcached = Memcached.new( ...servers..., ...options...)
 
-Place a YAML file in `config/memcached.yml` with contents like:
+Currently supported cache clients are: memcached, memcache-client
 
-    test:
-      ttl: 604800
-      namespace: ...
-      sessions: false
-      debug: false
-      servers: localhost:11211
-      cache_money: true
+#### Step 3: Configure Caching
 
-    development: 
-       ....
+Add the following to an initializer:
 
-#### Step 3: `config/environment.rb` ####
- config.gem "ngmoco-cache-money",
-   :lib => "cache_money",
-   :source => 'http://gems.github.com',
-   :version => '0.2.9'
+    Cash.configure :repository => $memcached, :adapter => :memcached
 
+Supported adapters are :memcache_client, :memcached. :memcached is assumed and is only compatible with Memcached clients.
+Local or transactional semantics may be disabled by setting :local => false or :transactional => false.
+
+Caching can be disabled on a per-environment basis in the environment's initializer:
+    
+    Cash.enabled = false
+    
 #### Step 4: Add indices to your ActiveRecord models ####
 
 Queries like `User.find(1)` will use the cache automatically. For more complex queries you must add indices on the attributes that you will query on. For example, a query like `User.find(:all, :conditions => {:name => 'bob'})` will require an index like:
@@ -187,24 +187,20 @@ There may be times where you only want to cache some of your models instead of e
 In that case, you can omit the following from your `config/initializers/cache_money.rb`
 
 	class ActiveRecord::Base
-	  is_cached :repository => $cache
+	  is_cached
 	end
 		
 After that is removed, you can simple put this at the top of your models you wish to cache:
 
-	is_cached :repository => $cache
-
-Just make sure that you put that line before any of your index directives.
-
-## Version ##
+	is_cached
 
-WARNING: This is currently a RELEASE CANDIDATE. A version of this code is in production use at Twitter but the extraction and refactoring process may have introduced bugs and/or performance problems. There are no known major defects at this point, but still.
+Just make sure that you put that line before any of your index directives. Note that all subclasses of a cached model are also cached.
 
 ## Acknowledgments ##
 
 Thanks to
 
  * Twitter for commissioning the development of this library and supporting the effort to open-source it.
- * Sam Luckenbill for pairing with me on most of the hard stuff.
+ * Sam Luckenbill for pairing with Nick on most of the hard stuff.
  * Matthew and Chris for pairing a few days, offering useful feedback on the readability of the code, and the initial implementation of the Memcached mock.
- * Evan Weaver for helping to reason-through software and testing strategies to deal with replication lag, and the initial implementation of the Memcached lock.
+ * Evan Weaver for helping to reason-through software and testing strategies to deal with replication lag, and the initial implementation of the Memcached lock.

data/README.markdown

@@ -117,9 +117,7 @@ Nested transactions are fully supported, with partial rollback and (apparent) pa
 
 ### Mocks ###
 
-For your unit tests, it is faster to use a Memcached mock than the real deal. Just place this in your initializer for your test environment:
-
-    $memcache = Cash::Mock.new
+For your unit tests, it is faster to use a Memcached mock than the real deal. In your test environment, initialize the repository with an instance of Cash::Mock.
     
 ### Locks ###
 
@@ -133,39 +131,41 @@ In most cases locks are unnecessary; the transactional Memcached client will tak
 
 Sometimes your code will request the same cache key twice in one request. You can avoid a round trip to the Memcached server by using a local, per-request cache. Add this to your initializer:
 
-  $local = Cash::Local.new($memcache)
-  $cache = Cash::Transactional.new($local, $lock)
-
+    $memcache = MemcachedWrapper.new(config[:servers].gsub(' ', '').split(','), config)
+    $local = Cash::Local.new($memcache)
+    $lock  = Cash::Lock.new($memcache)
+    $cache = Cash::Transactional.new($local, $lock)
+    
 ## Installation ##
 
-#### Step 0: Install MemCached
-
 #### Step 1: Get the GEM ####
 
-    % gem sources -a http://gems.github.com
     % sudo gem install ngmoco-cache-money
     
-#### Step 2: Configure MemCached.
+    Add the gem you your Gemfile:
+    gem 'ngmoco-cache-money', :lib => 'cache_money'
+    
+#### Step 2: Configure cache client
+
+In your environment, create a cache client instance configured for your cache servers.
+  
+    $memcached = Memcached.new( ...servers..., ...options...)
 
-Place a YAML file in `config/memcached.yml` with contents like:
+Currently supported cache clients are: memcached, memcache-client
 
-    test:
-      ttl: 604800
-      namespace: ...
-      sessions: false
-      debug: false
-      servers: localhost:11211
-      cache_money: true
+#### Step 3: Configure Caching
 
-    development: 
-       ....
+Add the following to an initializer:
 
-#### Step 3: `config/environment.rb` ####
- config.gem "ngmoco-cache-money",
-   :lib => "cache_money",
-   :source => 'http://gems.github.com',
-   :version => '0.2.9'
+    Cash.configure :repository => $memcached, :adapter => :memcached
 
+Supported adapters are :memcache_client, :memcached. :memcached is assumed and is only compatible with Memcached clients.
+Local or transactional semantics may be disabled by setting :local => false or :transactional => false.
+
+Caching can be disabled on a per-environment basis in the environment's initializer:
+    
+    Cash.enabled = false
+    
 #### Step 4: Add indices to your ActiveRecord models ####
 
 Queries like `User.find(1)` will use the cache automatically. For more complex queries you must add indices on the attributes that you will query on. For example, a query like `User.find(:all, :conditions => {:name => 'bob'})` will require an index like:
@@ -187,24 +187,20 @@ There may be times where you only want to cache some of your models instead of e
 In that case, you can omit the following from your `config/initializers/cache_money.rb`
 
 	class ActiveRecord::Base
-	  is_cached :repository => $cache
+	  is_cached
 	end
 		
 After that is removed, you can simple put this at the top of your models you wish to cache:
 
-	is_cached :repository => $cache
-
-Just make sure that you put that line before any of your index directives.
-
-## Version ##
+	is_cached
 
-WARNING: This is currently a RELEASE CANDIDATE. A version of this code is in production use at Twitter but the extraction and refactoring process may have introduced bugs and/or performance problems. There are no known major defects at this point, but still.
+Just make sure that you put that line before any of your index directives. Note that all subclasses of a cached model are also cached.
 
 ## Acknowledgments ##
 
 Thanks to
 
  * Twitter for commissioning the development of this library and supporting the effort to open-source it.
- * Sam Luckenbill for pairing with me on most of the hard stuff.
+ * Sam Luckenbill for pairing with Nick on most of the hard stuff.
  * Matthew and Chris for pairing a few days, offering useful feedback on the readability of the code, and the initial implementation of the Memcached mock.
- * Evan Weaver for helping to reason-through software and testing strategies to deal with replication lag, and the initial implementation of the Memcached lock.
+ * Evan Weaver for helping to reason-through software and testing strategies to deal with replication lag, and the initial implementation of the Memcached lock.

data/config/environment.rb

@@ -1,14 +1,6 @@
-require 'rubygems'
-gem 'activesupport', '~> 2.3.0'
-gem 'activerecord', '~> 2.3.0'
-gem 'actionpack', '~> 2.3.0'
-gem 'rspec', '>= 1.3.0'
-gem 'jeweler', '~> 1.4.0'
-
 require 'action_controller'
 require 'active_record'
 require 'active_record/session_store'
-require 'jeweler'
 
 ActiveRecord::Base.establish_connection(
   :adapter => 'sqlite3',

data/config/memcached.yml

@@ -1,6 +1,4 @@
 test:
   ttl: 604800
   namespace: cache
-  sessions: false
-  debug: false
   servers: localhost:11211

data/init.rb

@@ -1 +1 @@
-require File.join(File.basedir(__FILE__),'rails','init')
+require 'rails/init'

data/lib/cache_money.rb

@@ -1,6 +1,7 @@
 require 'active_support'
 require 'active_record'
 
+require 'cash/version'
 require 'cash/lock'
 require 'cash/transactional'
 require 'cash/write_through'
@@ -22,27 +23,31 @@ require 'cash/query/calculation'
 require 'cash/util/array'
 require 'cash/util/marshal'
 
-class ActiveRecord::Base
-  def self.is_cached(options = {})
-    if options == false
-      include NoCash
-    else
-      options.assert_valid_keys(:ttl, :repository, :version)
-      include Cash unless ancestors.include?(Cash)
-      Cash::Config.create(self, options)
-    end
-  end
-
-  def <=>(other)
-    if self.id == other.id then 
-      0
-    else
-      self.id < other.id ? -1 : 1
+module Cash
+  mattr_accessor :enabled
+  self.enabled = true
+  
+  mattr_accessor :repository
+  
+  def self.configure(options = {})
+    options.assert_valid_keys(:repository, :local, :transactional, :adapter, :default_ttl)
+    cache = options[:repository] || raise(":repository is a required option")
+    
+    adapter = options.fetch(:adapter, :memcached)
+    
+    if adapter
+      require "cash/adapter/#{adapter.to_s}"
+      klass = "Cash::Adapter::#{adapter.to_s.camelize}".constantize
+      cache = klass.new(cache, :logger => Rails.logger, :default_ttl => options.fetch(:default_ttl, 1.day.to_i))
     end
+    
+    lock  = Cash::Lock.new(cache)
+    cache = Cash::Local.new(cache) if options.fetch(:local, true)
+    cache = Cash::Transactional.new(cache, lock) if options.fetch(:transactional, true)
+    
+    self.repository = cache
   end
-end
-
-module Cash
+  
   def self.included(active_record_class)
     active_record_class.class_eval do
       include Config, Accessor, WriteThrough, Finders
@@ -50,6 +55,12 @@ module Cash
     end
   end
 
+  private
+
+    def self.repository
+      @@repository || raise("Cash.configure must be called when Cash.enabled is true")
+    end
+  
   module ClassMethods
     def self.extended(active_record_class)
       class << active_record_class
@@ -57,30 +68,38 @@ module Cash
       end
     end
 
-    def transaction_with_cache_transaction(*args)
-      if cache_config
-        transaction_without_cache_transaction(*args) do
-          repository.transaction { yield }
+    def transaction_with_cache_transaction(*args, &block)
+      if Cash.enabled
+        # Wrap both the db and cache transaction in another cache transaction so that the cache 
+        # gets written only after the database commit but can still flush the inner cache
+        # transaction if an AR::Rollback is issued.
+        Cash.repository.transaction do
+          transaction_without_cache_transaction(*args) do
+            Cash.repository.transaction { block.call }
+          end
         end
       else
-        transaction_without_cache_transaction(*args)
+        transaction_without_cache_transaction(*args, &block)
       end
     end
-
-    def cacheable?(*args)
-      true
-    end
   end
 end
-module NoCash
-  def self.included(active_record_class)
-    active_record_class.class_eval do
-      extend ClassMethods
-    end
+
+class ActiveRecord::Base
+  include Cash
+  
+  def self.is_cached(options = {})
+    options.assert_valid_keys(:ttl, :repository, :version)
+    opts = options.dup
+    opts[:repository] = Cash.repository unless opts.has_key?(:repository)
+    Cash::Config.create(self, opts)
   end
-  module ClassMethods
-    def cacheable?(*args)
-      false
+
+  def <=>(other)
+    if self.id == other.id then 
+      0
+    else
+      self.id < other.id ? -1 : 1
     end
   end
 end

data/lib/cash/adapter/memcache_client.rb

@@ -0,0 +1,36 @@
+require 'memcache'
+
+module Cash
+  module Adapter
+    class MemcacheClient
+      def initialize(repository, options = {})
+        @repository = repository
+        @logger = options[:logger]
+        @default_ttl = options[:default_ttl] || raise(":default_ttl is a required option")
+      end
+
+      def add(key, value, ttl=nil, raw=false)
+        @repository.add(key, value, ttl || @default_ttl, raw)
+      end
+      
+      def set(key, value, ttl=nil, raw=false)
+        @repository.set(key, value, ttl || @default_ttl, raw)
+      end
+      
+      def exception_classes
+        MemCache::MemCacheError
+      end
+      
+      def respond_to?(method)
+        super || @repository.respond_to?(method)
+      end
+      
+      private
+      
+        def method_missing(*args, &block)
+          @repository.send(*args, &block)
+        end
+        
+    end
+  end
+end

data/lib/cash/adapter/memcached.rb

@@ -0,0 +1,131 @@
+require 'memcached'
+
+# Maps memcached methods and semantics to those of memcache-client
+module Cash
+  module Adapter
+    class Memcached
+      def initialize(repository, options = {})
+        @repository = repository
+        @logger = options[:logger]
+        @default_ttl = options[:default_ttl] || raise(":default_ttl is a required option")   
+      end
+      
+      def add(key, value, ttl=nil, raw=false)
+        wrap(key, not_stored) do
+          logger.debug("Memcached add: #{key.inspect}") if debug_logger?
+          @repository.add(key, raw ? value.to_s : value, ttl || @default_ttl, !raw)
+          logger.debug("Memcached hit: #{key.inspect}") if debug_logger?
+          stored
+        end
+      end
+      
+      # Wraps Memcached#get so that it doesn't raise. This has the side-effect of preventing you from 
+      # storing <tt>nil</tt> values.
+      def get(key, raw=false)
+        wrap(key) do
+          logger.debug("Memcached get: #{key.inspect}") if debug_logger?
+          value = wrap(key) { @repository.get(key, !raw) }
+          logger.debug("Memcached hit: #{key.inspect}") if debug_logger?
+          value
+        end
+      end
+      
+      def get_multi(*keys)
+        wrap(keys, {}) do
+          begin
+            keys.flatten!
+            logger.debug("Memcached get_multi: #{keys.inspect}") if debug_logger?
+            values = @repository.get(keys, true)
+            logger.debug("Memcached hit: #{keys.inspect}") if debug_logger?
+            values
+          rescue TypeError
+            log_error($!) if logger
+            keys.each { |key| delete(key) }
+            logger.debug("Memcached deleted: #{keys.inspect}") if debug_logger?
+            {}
+          end
+        end
+      end
+      
+      def set(key, value, ttl=nil, raw=false)
+        wrap(key, not_stored) do
+          logger.debug("Memcached set: #{key.inspect}") if debug_logger?
+          @repository.set(key, raw ? value.to_s : value, ttl || @default_ttl, !raw)
+          logger.debug("Memcached hit: #{key.inspect}") if debug_logger?
+          stored
+        end
+      end
+      
+      def delete(key)
+        wrap(key, not_found) do
+          logger.debug("Memcached delete: #{key.inspect}") if debug_logger?
+          @repository.delete(key)
+          logger.debug("Memcached hit: #{key.inspect}") if debug_logger?
+          deleted
+        end
+      end
+      
+      def get_server_for_key(key)
+        wrap(key) { @repository.server_by_key(key) }
+      end
+
+      def incr(key, value = 1)
+        wrap(key) { @repository.incr(key, value) }
+      end
+
+      def decr(key, value = 1)
+        wrap(key) { @repository.decr(key, value) }
+      end
+      
+      def flush_all
+        @repository.flush
+      end
+      
+      def exception_classes
+        ::Memcached::Error
+      end
+      
+      private
+      
+        def logger
+          @logger
+        end
+        
+        def debug_logger?
+          logger && logger.respond_to?(:debug?) && logger.debug?
+        end
+        
+        def wrap(key, error_value = nil, options = {})
+          yield
+        rescue ::Memcached::NotStored
+          logger.debug("Memcached miss: #{key.inspect}") if debug_logger?
+          error_value
+        rescue ::Memcached::Error
+          log_error($!) if logger
+          raise if options[:reraise_error]
+          error_value
+        end
+        
+        def stored
+          "STORED\r\n"
+        end
+
+        def deleted
+          "DELETED\r\n"
+        end
+
+        def not_stored
+          "NOT_STORED\r\n"
+        end
+
+        def not_found
+          "NOT_FOUND\r\n"
+        end
+
+        def log_error(err)
+          #logger.error("#{err}: \n\t#{err.backtrace.join("\n\t")}") if logger
+          logger.error("Memcached ERROR, #{err.class}: #{err}") if logger
+        end
+    end
+  end
+end