arid_cache 1.2.0 → 1.3.0

data/README.rdoc

@@ -1,13 +1,14 @@
 = AridCache
 
-AridCache makes caching easy and effective.  AridCache supports caching on all of your ActiveRecord model named scopes, class and instance methods right out of the box.  AridCache keeps caching logic out of your model methods and clarifies your view code by making calls to cached result sets explicit.
+AridCache makes caching easy and effective.  AridCache supports caching on all ActiveRecord class and instance methods right out of the box.  AridCache keeps caching logic out of your model methods and clarifies your code by making calls to cached result sets explicit.
 
-AridCache supports caching large, expensive ActiveRecord collections by caching only the model IDs, provides efficient in-memory pagination of your cached collections, and gives you collection counts for free.  Non-ActiveRecord collection data is cached unchanged allowing you to cache the results of any expensive operation simply by prepending your method call with <tt>cached_</tt>.
+AridCache supports caching large, expensive ActiveRecord collections by caching only the model IDs, provides efficient in-memory pagination of your cached collections, and gives you collection counts for free.  Non-ActiveRecord collection data is cached unchanged allowing you to cache the results of anything simply by prepending your method call with <tt>cached_</tt>.
 
-AridCache simplifies caching by supporting auto-expiring cache keys - as well as common options like <tt>:expires_in</tt> - and provides methods to help you manage your caches at the global, model class, model instance and per-cache level.
+AridCache simplifies caching by supporting auto-expiring cache keys - as well as common options like <tt>:expires_in</tt> - and provides methods to help you manage your caches.
 
 == Changes
 
+v1.3.0: Support limits, ordering and pagination on cached Enumerables
 v1.2.0: Fix Rails 3 ActiveRecord hooks & remove some Rails dependencies
 v1.0.5: Support <tt>:raw</tt> and <tt>:clear</tt> options.
 
@@ -33,11 +34,27 @@ Then
 
   rake gems:install
 
+== Features
+
+* Include the AridCache module in any Class
+* Rails 2 & 3 compatible with automatic ActiveRecord::Base integration
+* Supports auto-expiring cache keys
+* Supports limits, ordering & pagination of cached Enumerables and ActiveRecord collections
+* Define caches and their options on your class using +instance_caches+ and +class_caches+
+* Counts for free - if you have already cached the result, you get the count for free
+* Supports eager-loading and other options to <tt>ActiveRecord::Base#find</tt> like
+:conditions, :include, :joins, :select, :readonly, :group, :having, :from
+* Provides methods to clear caches individually, at the instance-level, class-level and globally
+* Preserves the order of your cached ActiveRecord collections
+* Optimized to make as few cache and database accesses as absolutely neccessary
+
 == Introduction
 
 The name AridCache comes from <b>A</b>ctive<b>R</b>ecord *ID* Cache.  It's also very DRY...get it? :)
 
-Out of the box AridCache supports caching on all your ActiveRecord class and instance methods and named scopes...basically if a class or class instance <tt>respond_to?</tt> something, you can cache it.
+Out of the box AridCache supports caching on all your ActiveRecord class and instance methods and named scopes.  If a class or class instance <tt>respond_to?</tt> something, you can cache it.
+
+AridCache supports limits, pagination and ordering options on cached ActiveRecord collections and any other Enumerable.  Options to apply limits are <tt>:limit</tt> and <tt>:offset</tt>.  Options for pagination are <tt>:page</tt> and <tt>:per_page</tt> and the <tt>:order</tt> option accepts the same values as ActiveRecord::Base#find.  If the cached value is an ActiveRecord collection most other options to ActiveRecord::Base#find are supported too.  If the cached value is an Enumerable (e.g. an Array) the value for :order must be a <tt>Proc</tt>.  The Proc is passed to Enumerable#sort to do the sorting.  Unless the Enumerable is a list of Hashes in which case it can be a String or Symbol giving the hash key to sort by.
 
 The way you interact with the cache via your model methods is to prepend the method call with <tt>cached_</tt>.  The part of the method call after <tt>cached_</tt> serves as the basis for the cache key.  For example,
 
@@ -46,7 +63,6 @@ The way you interact with the cache via your model methods is to prepend the met
 
 You can also define caches that use compositions of methods or named scopes, or other complex queries, without having to add a new method to your class.  This way you can also create different caches that all use the same method.  For example,
 
-  # cache key is arid-cache-user-most_active_users
   User.cached_most_active_users do
     active.find(:order => 'activity DESC', :limit => 5)
   end
@@ -55,17 +71,19 @@ You can also define caches that use compositions of methods or named scopes, or
 
 If the result of your <tt>cached_</tt> call is an array of ActiveRecords, AridCache only stores the IDs in the cache (because it's a bad idea to store records in the cache).
 
-On subsequent calls we call <tt>find_all_by_id</tt> on the target class passing in the ActiveRecord IDs that were stored in the cache.  AridCache will preserve the original ordering of your collection (you can change this using the <tt>:order</tt>).
+On subsequent calls we call <tt>find_all_by_id</tt> on the target class passing in the ActiveRecord IDs that were stored in the cache.  AridCache will preserve the original ordering of your collection (you can change this using the <tt>:order</tt> option).
 
 The idea here is to cache collections that are expensive to query.  Once the cache is loaded, retrieving the cached records from the database simply involves a <tt>SELECT * FROM table WHERE id IN (ids, ...)</tt>.
 
 Consider how long it would take to get the top 10 favorited tracks of all time from a database with a million tracks and 100,000 users.  Now compare that to selecting 10 tracks by ID from the track table.  The performance gain is huge.
 
-=== Base Types and Other Collections
+=== Enumerables
+
+Cached enumerables support find-like options such as <tt>:limit</tt>, <tt>:offset</tt> and <tt>order</tt> as well as pagination options <tt>:page</tt> and <tt>:per_page</tt>.  <tt>:order</tt> must be a <tt>Proc</tt>.  It is passed to Enumerable#sort to do the sorting.  Unless the enumerable contains hashes in which case <tt>:order</tt> can be a string or symbol hash key to order by.
 
-Arrays of non-ActiveRecords are stored as-is so you can cache arrays of strings and other types without problems.
+=== Base Types and Other Collections
 
-Any other objects (including single ActiveRecord objects) are cached and returned as-is.
+Anything that is not an array of ActiveRecords is cached as-is.  Numbers, arrays, hashes, nils, whatever.
 
 === Example
 

data/VERSION

@@ -1 +1 @@
-1.2.0
+1.3.0

data/arid_cache.gemspec

@@ -5,11 +5,11 @@
 
 Gem::Specification.new do |s|
   s.name = %q{arid_cache}
-  s.version = "1.2.0"
+  s.version = "1.3.0"
 
   s.required_rubygems_version = Gem::Requirement.new(">= 0") if s.respond_to? :required_rubygems_version=
   s.authors = ["Karl Varga"]
-  s.date = %q{2011-03-28}
+  s.date = %q{2011-04-05}
   s.description = %q{AridCache makes caching easy and effective.  AridCache supports caching on all your model named scopes, class methods and instance methods right out of the box.  AridCache prevents caching logic from cluttering your models and clarifies your logic by making explicit calls to cached result sets.
 AridCache is designed for handling large, expensive ActiveRecord collections but is equally useful for caching anything else as well.
 }
@@ -30,6 +30,9 @@ AridCache is designed for handling large, expensive ActiveRecord collections but
      "lib/arid_cache.rb",
      "lib/arid_cache/active_record.rb",
      "lib/arid_cache/cache_proxy.rb",
+     "lib/arid_cache/cache_proxy/options.rb",
+     "lib/arid_cache/cache_proxy/result_processor.rb",
+     "lib/arid_cache/cache_proxy/utilities.rb",
      "lib/arid_cache/helpers.rb",
      "lib/arid_cache/inflector.rb",
      "lib/arid_cache/inflector/inflections.rb",
@@ -38,7 +41,9 @@ AridCache is designed for handling large, expensive ActiveRecord collections but
      "rails/init.rb",
      "spec/arid_cache/active_record_spec.rb",
      "spec/arid_cache/arid_cache_spec.rb",
-     "spec/arid_cache/cache_proxy_result_spec.rb",
+     "spec/arid_cache/cache_proxy/cached_result_spec.rb",
+     "spec/arid_cache/cache_proxy/options_spec.rb",
+     "spec/arid_cache/cache_proxy/result_processor_spec.rb",
      "spec/arid_cache/cache_proxy_spec.rb",
      "spec/spec.opts",
      "spec/spec_helper.rb",
@@ -66,7 +71,9 @@ AridCache is designed for handling large, expensive ActiveRecord collections but
   s.test_files = [
     "spec/arid_cache/active_record_spec.rb",
      "spec/arid_cache/arid_cache_spec.rb",
-     "spec/arid_cache/cache_proxy_result_spec.rb",
+     "spec/arid_cache/cache_proxy/cached_result_spec.rb",
+     "spec/arid_cache/cache_proxy/options_spec.rb",
+     "spec/arid_cache/cache_proxy/result_processor_spec.rb",
      "spec/arid_cache/cache_proxy_spec.rb",
      "spec/spec_helper.rb",
      "spec/support/ar_query.rb",

data/lib/arid_cache/cache_proxy/options.rb

@@ -0,0 +1,73 @@
+module AridCache
+  class CacheProxy
+    # A class representing a hash of options with methods to return subsets of
+    # those options.
+    class Options < Hash
+      def initialize(opts={})
+        self.merge!(opts)
+      end
+      
+      # Filter options for paginate.  Get the :per_page value from the receiver if it's not set.
+      # Set total_entries to +records.size+ if +records+ is supplied
+      def opts_for_paginate(records=nil)
+        paginate_opts = reject { |k,v| !OPTIONS_FOR_PAGINATE.include?(k) }
+        paginate_opts[:finder] = :find_all_by_id unless paginate_opts.include?(:finder)
+        if self[:result_klass].respond_to?(:per_page) && !paginate_opts.include?(:per_page)
+          paginate_opts[:per_page] = self[:result_klass].per_page 
+        end
+        paginate_opts[:total_entries] = records.size unless records.nil?
+        paginate_opts
+      end
+
+      # Return options suitable to pass to ActiveRecord::Base#find.
+      # Preserve the original order of the results if no :order option is specified.
+      # If an offset is specified but no limit, ActiveRecord will not apply the offset,
+      # so pass in a limit that is as big as +ids.size+
+      #
+      # @arg ids array of ids to order by unless an :order option is specified.
+      def opts_for_find(ids)
+        find_opts = reject { |k,v| !OPTIONS_FOR_FIND.include?(k) }
+        find_opts[:order] = AridCache::CacheProxy::Utilities.order_by(ids, self[:result_klass]) unless find_opts.include?(:order)
+        find_opts[:limit] = ids.size unless find_opts.include?(:limit)
+        find_opts
+      end
+
+      def opts_for_cache
+        reject { |k,v| !OPTIONS_FOR_CACHE.include?(k) }
+      end
+
+      def opts_for_cache_key
+        reject { |k,v| !OPTIONS_FOR_CACHE_KEY.include?(k) }
+      end
+
+      # Returns options that affect the cache proxy result
+      def opts_for_cache_proxy
+        reject { |k,v| !OPTIONS_FOR_CACHE_PROXY.include?(k) }
+      end  
+      
+      def force?
+        !!self[:force]
+      end 
+      
+      def paginate?
+        include?(:page)
+      end
+      
+      def raw?
+        !!self[:raw]
+      end
+      
+      def count_only?
+        !!self[:count_only]
+      end
+    
+      def order_by_proc?
+        include?(:order) && self[:order].is_a?(Proc)
+      end
+      
+      def order_by_key?
+        include?(:order) && (self[:order].is_a?(Symbol) || self[:order].is_a?(String))
+      end
+    end
+  end
+end

data/lib/arid_cache/cache_proxy/result_processor.rb

@@ -0,0 +1,199 @@
+module AridCache
+  class CacheProxy
+    # A class representing a result that is to be processed in some way before
+    # being returned to the user.
+    #
+    # Provides methods to introspect the result.  The contents could be a base type,
+    # or an enumerable of sorts...any type really.  We are only concerned with enumerables,
+    # and especially those containing active records.
+    class ResultProcessor
+
+      def initialize(result, opts={})
+        @result = result
+        @options = opts.is_a?(AridCache::CacheProxy::Options) ? opts : AridCache::CacheProxy::Options.new(opts)
+      end
+
+      # Return true if the result is an enumerable and it is empty.
+      def is_empty?
+        is_enumerable? && @result.empty?
+      end
+
+      # Return true if the result is an enumerable.
+      def is_enumerable?
+        @result.is_a?(Enumerable)
+      end
+
+      # Return true if the result is a list of hashes
+      def is_hashes?
+        is_enumerable? && @result.first.is_a?(Hash)
+      end
+
+      # Order in the database if an order clause has been specified and we
+      # have a list of ActiveRecords or a CachedResult.
+      def order_in_database?
+        is_cached_result? || (@options.order_by_key? && is_activerecord?)
+      end
+
+      # Return true if the result is an enumerable and the first item is
+      # an active record.
+      def is_activerecord?
+        is_enumerable? && @result.first.is_a?(::ActiveRecord::Base)
+      end
+
+      def is_activerecord_reflection?
+        @result.respond_to?(:proxy_reflection) || @result.respond_to?(:proxy_options)
+      end
+
+      def is_cached_result?
+        @result.is_a?(AridCache::CacheProxy::CachedResult)
+      end
+
+      # Return the result to cache.  For base types the original result is
+      # returned.  ActiveRecords return a CachedResult.
+      def to_cache
+        # Check if it's an association first, because it doesn't trigger the select if it's
+        # a named scope.  Calling respond_to? on an association proxy will trigger a select
+        # because it loads up the target and passes the respond_to? on to it.
+        @cached = if is_activerecord_reflection?
+          lazy_cache.klass = @result.proxy_reflection.klass if @result.respond_to?(:proxy_reflection)
+          if @options.count_only?
+            lazy_cache.count = @result.count
+          else
+            lazy_cache.ids = @result.collect { |r| r[:id] }
+            lazy_cache.count = @result.size
+          end
+          lazy_cache
+        elsif is_activerecord? || is_empty?
+          lazy_cache.ids = @result.collect { |r| r[:id] }
+          lazy_cache.count = @result.size
+          lazy_cache.klass = @result.first.class
+          lazy_cache
+        else
+          @result
+        end
+      end
+
+      # Apply any options like pagination or ordering and return the result, which
+      # is either some base type, or usually, a list of active records.
+      def to_result
+        if @options.count_only?
+          get_count
+        elsif @options.raw? || (!is_cached_result? && !is_enumerable?)
+          @result
+        else
+          if is_cached_result?
+            fetch_activerecords(filter_results(@result.ids))
+          elsif order_in_database?
+            fetch_activerecords(filter_results(@result))
+          else
+            filter_results(@result)
+          end
+        end
+      end
+
+      private
+
+      def get_count
+        if @cached.is_a?(AridCache::CacheProxy::CachedResult) # use what we put in the cache
+          @cached.count
+        elsif @result.respond_to?(:count)
+          @result.count
+        else
+          @result
+        end
+      end
+
+      # Lazy-initialize a new cached result.  Default the klass of the result to
+      # that of the receiver.
+      def lazy_cache
+        return @lazy_cache if @lazy_cache
+        @lazy_cache = AridCache::CacheProxy::CachedResult.new
+        @lazy_cache.klass = @options[:result_klass]
+        @lazy_cache
+      end
+
+      # Return the result after processing it to apply limits or pagination in memory.
+      # Doesn't do anything if we have to order in the databse.
+      #
+      # Options are only applied if the object responds to the appropriate method.
+      # So for example pagination will not happen unless the object responds to :paginate.
+      #
+      # Options:
+      #   :order  - Ordering is done first, before applying limits or paginating.
+      #             If it's a Proc it is passed to Array#sort to do the sorting.
+      #             If it is a Symbol or String the results should be Hashes and the
+      #             list of Hashes are sorted by the values at the given key.
+      #   :limit  - limit the array to the specified size
+      #   :offset - ignore the first +offset+ items in the array
+      #   :page / :per_page - paginate the result.  If :limit is specified, the array is
+      #             limited before paginating; similarly if :offset is specified the array is offset
+      #             before paginating.  Pagination only happens if the :page option is passed.
+      def filter_results(records)
+        return records if order_in_database?
+
+        # Order in memory
+        if records.respond_to?(:sort)
+          if @options.order_by_proc?
+            records = records.sort(&@options[:order])
+          elsif @options.order_by_key? && is_hashes?
+            records = records.sort do |a, b|
+              a[@options[:order]] <=> b[@options[:order]]
+            end
+          end
+        end
+
+        # Limit / Offset
+        if (@options.include?(:offset) || @options.include?(:limit)) && records.respond_to?(:[])
+          records = records[@options[:offset] || 0, @options[:limit] || records.size]
+        end
+
+        # Paginate
+        if @options.paginate? && records.respond_to?(:paginate)
+          # Convert records to an array before calling paginate.  If we don't do this
+          # and the result is a named scope, paginate will trigger an additional query
+          # to load the page rather than just using the records we have already fetched.
+          records = records.respond_to?(:to_a) ? records.to_a : records
+          records = records.paginate(@options.opts_for_paginate(records))
+        end
+        records
+      end
+
+      # Return a list of records from the database.  +records+ is a list of
+      # ActiveRecords or a list of ActiveRecord ids.
+      #
+      # If no :order is specified, the current order is preserved with some fancy SQL.
+      # If an arder is specified then
+      # order, limit and paginate in the database.
+      def fetch_activerecords(records)
+        if records.empty?
+          if @options.paginate?
+            return records.paginate(@options.opts_for_paginate(records))
+          else
+            return records
+          end
+        end
+
+        ids = records.first.is_a?(ActiveRecord) ? records.collect { |record| record[:id] } : records
+        find_opts = @options.opts_for_find(ids)
+        if order_in_database?
+          if @options.paginate?
+            find_opts.merge!(@options.opts_for_paginate(ids))
+            result_klass.paginate(ids, find_opts)
+          else
+            result_klass.find_all_by_id(ids, find_opts)
+          end
+        else
+          # Limits will have already been applied, remove them from the options for find.
+          [:offset, :limit].each { |key| find_opts.delete(key) }
+          result = result_klass.find_all_by_id(ids, find_opts)
+          records.is_a?(::WillPaginate::Collection) ? records.replace(result) : result
+        end
+      end
+
+      # Return the klass to use for building results (only applies to ActiveRecord results)
+      def result_klass
+        @options[:result_klass] = is_cached_result? ? @result.klass : (@cached.is_a?(AridCache::CacheProxy::CachedResult) ? @cached.klass : Utilities.object_class(@options[:receiver]))
+      end
+    end
+  end
+end

data/lib/arid_cache/cache_proxy/utilities.rb

@@ -0,0 +1,35 @@
+module AridCache
+  class CacheProxy
+    module Utilities
+      extend self
+
+      # Generate an ORDER BY clause that preserves the ordering of the ids in *ids*.
+      #
+      # The method we use depends on the database adapter because only MySQL
+      # supports the ORDER BY FIELD() function.  For other databases we use
+      # a CASE statement.
+      def order_by(ids, klass=nil)
+        column = if klass.respond_to?(:table_name)
+          ::ActiveRecord::Base.connection.quote_table_name(klass.table_name) + '.id'
+        else
+          "id"
+        end
+
+        if ids.empty?
+          nil
+        elsif ::ActiveRecord::Base.is_mysql_adapter?
+          "FIELD(#{column},#{ids.join(',')})"
+        else
+          order = ''
+          ids.each_index { |i| order << "WHEN #{column}=#{ids[i]} THEN #{i+1} " }
+          "CASE " + order + " END"
+        end
+      end
+
+      # Return the object's class or the object if it is a class.
+      def object_class(object)
+        object.is_a?(Class) ? object : object.class
+      end
+    end
+  end
+end