api_cache 0.1.2 → 0.2.0

data/README.rdoc

@@ -0,0 +1,94 @@
+= APICache (aka api_cache)
+
+APICache allows any API client library to be easily wrapped with a robust caching layer. It supports caching (obviously), serving stale data and limits on the number of API calls. It's also got a handy syntax if all you want to do is cache a bothersome url.
+
+== For the impatient
+
+    # Install
+    sudo gem install api_cache -s http://gemcutter.org
+    
+    # Require
+    require 'rubygems'
+    require 'api_cache'
+    
+    # Use
+    APICache.get("http://twitter.com/statuses/public_timeline.rss")
+    
+    # Use a proper store
+    require 'moneta/memcache'
+    APICache.store = Moneta::Memcache.new(:server => "localhost")
+    
+    # Wrap an API, and handle the failure case
+    
+    APICache.get("my_albums", :fail => []) do
+      FlickrRb.get_all_sets
+    end
+
+== The longer version
+
+You want to use the Twitter API but you don't want to die?
+
+    APICache.get("http://twitter.com/statuses/public_timeline.rss")
+
+This works better than a standard HTTP get because you get the following functionality for free:
+
+* Cached response returned for 10 minutes
+* Stale response returned for a day if twitter is down
+* Limited to attempt a connection at most once a minute
+
+To understand what <tt>APICache</tt> does here's an example: Given cached data less than 10 minutes old, it returns that. Otherwise, assuming it didn't try to request the URL within the last minute (to avoid the rate limit), it makes a get request to the supplied url. If the Twitter API timeouts or doesn't return a 2xx code (very likely) we're still fine: it just returns the last data fetched (as long as it's less than a day old). In the exceptional case that all is lost and no data can be returned, a subclass of <tt>APICache::APICacheError</tt> is raised which you're responsible for rescuing.
+
+Assuming that you don't care whether it was a timeout error or an invalid response (for example) you could do this:
+
+    begin
+      APICache.get("http://twitter.com/statuses/public_timeline.rss")
+    rescue APICache::APICacheError
+      "Fail Whale"
+    end
+
+However there's an easier way if you don't care exactly why the API call failed. You can just pass the :fail parameter (procs are accepted too) and all exceptions will be rescued for you. So this is exactly equivalent:
+
+    APICache.get("http://twitter.com/statuses/public_timeline.rss", {
+      :fail => "Fail Whale"
+    })
+
+The real value however is not caching HTTP calls, but allowing caching functionality to be easily added to existing API client gems, or in fact any arbitrary code which is either slow or not guaranteed to succeed every time.
+
+    APICache.get('twitter_replies', :cache => 3600) do
+      Net::HTTP.start('twitter.com') do |http|
+        req = Net::HTTP::Get.new('/statuses/replies.xml')
+        req.basic_auth 'username', 'password'
+        response = http.request(req)
+        case response
+        when Net::HTTPSuccess
+          # 2xx response code
+          response.body
+        else
+          raise APICache::InvalidResponse
+        end
+      end
+    end
+
+The first argument to <tt>APICache.get</tt> is now assumed to be a unique key rather than a URL. As you'd expect, the block will only be called if the request cannot be fulfilled by the cache. Throwing any exception signals to <tt>APICache</tt> that the request was not successful, should not be cached, and a cached value should be returned if available. If a cached value is not available then the exception will be re-raised for you to handle.
+
+You can send any of the following options to <tt>APICache.get(url, options = {}, &block)</tt>. These are the default values (times are all in seconds):
+
+    {
+      :cache => 600,    # 10 minutes  After this time fetch new data
+      :valid => 86400,  # 1 day       Maximum time to use old data
+                        #             :forever is a valid option
+      :period => 60,    # 1 minute    Maximum frequency to call API
+      :timeout => 5     # 5 seconds   API response timeout
+      :fail =>          # Value returned instead of exception on failure
+    }
+
+Before using the APICache you should set the cache to use. By default an in memory hash is used - obviously not a great idea. Thankfully APICache can use any moneta store, so for example if you wanted to use memcache you'd do this:
+
+    require 'moneta/memcache'
+    APICache.store = Moneta::Memcache.new(:server => "localhost")
+
+Please be liberal with the github issue tracker, more so with pull requests, or drop me a mail to me [at] mloughran [dot] com. I'd love to hear from you.
+
+== Copyright
+
+Copyright (c) 2008 Martyn Loughran. See LICENSE for details.

data/VERSION.yml

@@ -1,4 +1,4 @@
 --- 
-:patch: 2
+:patch: 0
 :major: 0
-:minor: 1
+:minor: 2

data/lib/api_cache.rb

@@ -1,64 +1,128 @@
+require 'logger'
+
+# Contains the complete public API for APICache.
+#
+# See APICache.get method and the README file.
+#
+# Before using APICache you should set the store to use using
+# APICache.store=. For convenience, when no store is set an in memory store
+# will be used (and a warning will be logged).
+#
 class APICache
-  class NotAvailableError < RuntimeError; end
-  class Invalid <  RuntimeError; end
-  class CannotFetch < RuntimeError; end
-  
+  class APICacheError < RuntimeError; end
+  class NotAvailableError < APICacheError; end
+  class TimeoutError < NotAvailableError; end
+  class InvalidResponse <  NotAvailableError; end
+  class CannotFetch < NotAvailableError; end
+
   class << self
-    attr_accessor :cache
-    attr_accessor :api
     attr_accessor :logger
+    attr_accessor :store
+
+    def logger # :nodoc:
+      @logger ||= begin
+        log = Logger.new(STDOUT)
+        log.level = Logger::INFO
+        log
+      end
+    end
+
+    # Set the logger to use. If not set, <tt>Logger.new(STDOUT)</tt> will be
+    # used.
+    #
+    def logger=(logger)
+      @logger = logger
+    end
+
+    def store # :nodoc:
+      @store ||= begin
+        APICache.logger.warn("Using in memory store")
+        APICache::MemoryStore.new
+      end
+    end
+
+    # Set the cache store to use. This should either be an instance of a
+    # moneta store or a subclass of APICache::AbstractStore. Moneta is
+    # recommended.
+    #
+    def store=(store)
+      @store = begin
+        if store.class < APICache::AbstractStore
+          store
+        elsif store.class.to_s =~ /Moneta/
+          MonetaStore.new(store)
+        elsif store.nil?
+          nil
+        else
+          raise ArgumentError, "Please supply an instance of either a moneta store or a subclass of APICache::AbstractStore"
+        end
+      end
+    end
   end
-  
-  # Initializes the cache
-  def self.start(store = APICache::MemcacheStore, logger = APICache::Logger.new)
-    APICache.logger = logger
-    APICache.cache  = APICache::Cache.new(store)
-    APICache.api    = APICache::API.new
-  end
-  
-  # Raises an APICache::NotAvailableError if it can't get a value. You should rescue this
-  # if your application code.
-  # 
-  # Optionally call with a block. The value of the block is then used to 
+
+  # Optionally call with a block. The value of the block is then used to
   # set the cache rather than calling the url. Use it for example if you need
   # to make another type of request, catch custom error codes etc. To signal
-  # that the call failed just raise APICache::Invalid - the value will then 
-  # not be cached and the api will not be called again for options[:timeout] 
-  # seconds. If an old value is available in the cache then it will be used.
-  # 
+  # that the call failed just raise any exception - the value will then
+  # not be cached and the api will not be called again for options[:timeout]
+  # seconds. If an old value is available in the cache then it will be
+  # returned.
+  #
+  # An exception will be raised if the API cannot be fetched and the request
+  # cannot be served by the cache. This will either be a subclass of
+  # APICache::Error or an exception raised by the provided block.
+  #
   # For example:
   #   APICache.get("http://twitter.com/statuses/user_timeline/6869822.atom")
-  # 
+  #
   #   APICache.get \
   #     "http://twitter.com/statuses/user_timeline/6869822.atom",
   #     :cache => 60, :valid => 600
+  #
   def self.get(key, options = {}, &block)
     options = {
       :cache => 600,    # 10 minutes  After this time fetch new data
       :valid => 86400,  # 1 day       Maximum time to use old data
-                        #             :forever is a valid option
+      #             :forever is a valid option
       :period => 60,    # 1 minute    Maximum frequency to call API
       :timeout => 5     # 5 seconds   API response timeout
     }.merge(options)
-    
-    cache_state = cache.state(key, options[:cache], options[:valid])
-    
+
+    cache = APICache::Cache.new(key, {
+      :cache => options[:cache],
+      :valid => options[:valid]
+    })
+
+    api = APICache::API.new(key, {
+      :period => options[:period],
+      :timeout => options[:timeout]
+    }, &block)
+
+    cache_state = cache.state
+
     if cache_state == :current
-      cache.get(key)
+      cache.get
     else
       begin
-        raise APICache::CannotFetch unless api.queryable?(key, options[:period])
-        value = api.get(key, options[:timeout], &block)
-        cache.set(key, value)
+        value = api.get
+        cache.set(value)
         value
-      rescue APICache::CannotFetch => e
-        APICache.logger.log "Failed to fetch new data from API because " \
+      rescue => e
+        APICache.logger.info "Failed to fetch from API - Exception: " \
           "#{e.class}: #{e.message}"
+        # No point outputting backgraces for internal APICache errors
+        APICache.logger.debug "Backtrace:\n#{e.backtrace.join("\n")}" unless e.kind_of?(APICacheError)
+
         if cache_state == :refetch
-          cache.get(key)
+          cache.get
         else
-          APICache.logger.log "Data not available in the cache or from API"
-          raise APICache::NotAvailableError, e.message
+          APICache.logger.warn "Data not available in the cache or from API for key #{@key}"
+          if options.has_key?(:fail)
+            fail = options[:fail]
+            fail.respond_to?(:call) ? fail.call : fail
+          else
+            raise e
+          end
         end
       end
     end
@@ -67,8 +131,7 @@ end
 
 require 'api_cache/cache'
 require 'api_cache/api'
-require 'api_cache/logger'
 
 APICache.autoload 'AbstractStore', 'api_cache/abstract_store'
 APICache.autoload 'MemoryStore', 'api_cache/memory_store'
-APICache.autoload 'MemcacheStore', 'api_cache/memcache_store'
+APICache.autoload 'MonetaStore', 'api_cache/moneta_store'

data/lib/api_cache/abstract_store.rb

@@ -1,25 +1,27 @@
-class APICache::AbstractStore
-  def initialize
-    raise "Method not implemented. Called abstract class."
-  end
-  
-  # Set value. Returns true if success.
-  def set(key, value)
-    raise "Method not implemented. Called abstract class."
-  end
-  
-  # Get value.
-  def get(key)
-    raise "Method not implemented. Called abstract class."
-  end
-  
-  # Does a given key exist in the cache?
-  def exists?(key)
-    raise "Method not implemented. Called abstract class."
-  end
-  
-  # Has a given time passed since the key was set?
-  def expired?(key, timeout)
-    raise "Method not implemented. Called abstract class."
+class APICache
+  class AbstractStore
+    def initialize
+      raise "Method not implemented. Called abstract class."
+    end
+
+    # Set value. Returns true if success.
+    def set(key, value)
+      raise "Method not implemented. Called abstract class."
+    end
+
+    # Get value.
+    def get(key)
+      raise "Method not implemented. Called abstract class."
+    end
+
+    # Does a given key exist in the cache?
+    def exists?(key)
+      raise "Method not implemented. Called abstract class."
+    end
+
+    # Has a given time passed since the key was set?
+    def expired?(key, timeout)
+      raise "Method not implemented. Called abstract class."
+    end
   end
 end

data/lib/api_cache/api.rb

@@ -1,70 +1,97 @@
 require 'net/http'
 
-# This class wraps up querying the API and remembers when each API was 
-# last queried in case there is a limit to the number that can be made.
-class APICache::API
-  def initialize
-    @query_times = {}
-  end
-  
-  # Checks whether the API can be queried (i.e. whether retry_time has passed
-  # since the last query to the API).
-  # 
-  # If retry_time is 0 then there is no limit on how frequently queries can
-  # be made to the API.
-  def queryable?(key, retry_time)
-    if @query_times[key]
-      if Time.now - @query_times[key] > retry_time
-        APICache.logger.log "Queryable: true - retry_time has passed"
-        true
+class APICache
+  # Wraps up querying the API.
+  #
+  # Ensures that the API is not called more frequently than every +period+
+  # seconds, and times out API requests after +timeout+ seconds.
+  #
+  class API
+    # Takes the following options
+    #
+    # period:: Maximum frequency to call the API. If set to 0 then there is no
+    #          limit on how frequently queries can be made to the API.
+    # timeout:: Timeout when calling api (either to the proviced url or
+    #           excecuting the passed block)
+    # block:: If passed then the block is excecuted instead of HTTP GET
+    #         against the provided key
+    #
+    def initialize(key, options, &block)
+      @key, @block = key, block
+      @timeout = options[:timeout]
+      @period = options[:period]
+    end
+
+    # Fetch data from the API.
+    #
+    # If no block is given then the key is assumed to be a URL and which will
+    # be queried expecting a 200 response. Otherwise the return value of the
+    # block will be used.
+    #
+    # This method can raise Timeout::Error, APICache::InvalidResponse, or any
+    # exception raised in the block passed to APICache.get
+    #
+    def get
+      check_queryable!
+      APICache.logger.debug "Fetching data from the API"
+      set_queried_at
+      Timeout::timeout(@timeout) do
+        if @block
+          # If this call raises an error then the response is not cached
+          @block.call
+        else
+          get_key_via_http
+        end
+      end
+    rescue Timeout::Error => e
+      raise APICache::TimeoutError, "Timed out when calling API (timeout #{@timeout}s)"
+    end
+
+    private
+
+    def get_key_via_http
+      response = redirecting_get(@key)
+      case response
+      when Net::HTTPSuccess
+        # 2xx response code
+        response.body
       else
-        APICache.logger.log "Queryable: false - queried too recently"
-        false
+        raise APICache::InvalidResponse, "InvalidResponse http response: #{response.code}"
       end
-    else
-      APICache.logger.log "Queryable: true - never used API before"
-      true
     end
-  end
-  
-  # Fetch data from the API.
-  # 
-  # If no block is given then the key is assumed to be a URL and which will
-  # be queried expecting a 200 response. Otherwise the return value of the
-  # block will be used.
-  # 
-  # If the block is unable to fetch the value from the API it should raise
-  # APICache::Invalid.
-  def get(key, timeout, &block)
-    APICache.logger.log "Fetching data from the API"
-    @query_times[key] = Time.now
-    Timeout::timeout(timeout) do
-      if block_given?
-        # This should raise APICache::Invalid if it is not correct
-        yield
+
+    def redirecting_get(url)
+      r = Net::HTTP.get_response(URI.parse(url))
+      r.header['location'] ? redirecting_get(r.header['location']) : r
+    end
+
+    # Checks whether the API can be queried (i.e. whether :period has passed
+    # since the last query to the API).
+    #
+    def check_queryable!
+      if previously_queried?
+        if Time.now - queried_at > @period
+          APICache.logger.debug "Queryable: true - retry_time has passed"
+        else
+          APICache.logger.debug "Queryable: false - queried too recently"
+          raise APICache::CannotFetch,
+            "Cannot fetch #{@key}: queried too recently"
+        end
       else
-        get_via_http(key, timeout)
+        APICache.logger.debug "Queryable: true - never used API before"
       end
     end
-  rescue Timeout::Error, APICache::Invalid => e
-    raise APICache::CannotFetch, e.message
-  end
-  
-private
-  
-  def get_via_http(key, timeout)
-    response = redirecting_get(key)
-    case response
-    when Net::HTTPSuccess
-      # 2xx response code
-      response.body
-    else
-      raise APICache::Invalid, "Invalid http response: #{response.code}"
+
+    def previously_queried?
+      APICache.store.exists?("#{@key}_queried_at")
+    end
+
+    def queried_at
+      APICache.store.get("#{@key}_queried_at")
     end
-  end
 
-  def redirecting_get(url)
-    r = Net::HTTP.get_response(URI.parse(url))
-    r.header['location'] ? redirecting_get(r.header['location']) : r
+    def set_queried_at
+      APICache.store.set("#{@key}_queried_at", Time.now)
+    end
   end
 end

data/lib/api_cache/cache.rb

@@ -1,41 +1,60 @@
-# Cache performs calculations relating to the status of items stored in the
-# cache and delegates storage to the various cache stores.
 require 'digest/md5'
-class APICache::Cache
-  attr_accessor :store
-  def initialize(store)
-    @store = store.send(:new)
-  end
-  
-  # Returns one of the following options depending on the state of the key:
-  # 
-  # * :current (key has been set recently)
-  # * :refetch (data should be refetched but is still available for use)
-  # * :invalid (data is too old to be useful)
-  # * :missing (do data for this key)
-  def state(key, refetch_time, invalid_time)
-    if @store.exists?(encode(key))
-      if !@store.expired?(encode(key), refetch_time)
-        :current
-      elsif (invalid_time == :forever) || !@store.expired?(encode(key), invalid_time)
-        :refetch
+
+class APICache
+  # Cache performs calculations relating to the status of items stored in the
+  # cache and delegates storage to the various cache stores.
+  #
+  class Cache
+    # Takes the following options
+    #
+    # cache:: Length of time to cache before re-requesting
+    # valid:: Length of time to consider data still valid if API cannot be
+    #         fetched - :forever is a valid option.
+    #
+    def initialize(key, options)
+      @key = key
+      @cache = options[:cache]
+      @valid = options[:valid]
+    end
+
+    # Returns one of the following options depending on the state of the key:
+    #
+    # * :current (key has been set recently)
+    # * :refetch (data should be refetched but is still available for use)
+    # * :invalid (data is too old to be useful)
+    # * :missing (do data for this key)
+    #
+    def state
+      if store.exists?(hash)
+        if !store.expired?(hash, @cache)
+          :current
+        elsif (@valid == :forever) || !store.expired?(hash, @valid)
+          :refetch
+        else
+          :invalid
+        end
       else
-        :invalid
+        :missing
       end
-    else
-      :missing
     end
-  end
-  
-  def get(key)
-    @store.get(encode(key))
-  end
-  
-  def set(key, value)
-    @store.set(encode(key), value)
-    true
-  end
-  def encode(key)
-    Digest::MD5.hexdigest key
+
+    def get
+      store.get(hash)
+    end
+
+    def set(value)
+      store.set(hash, value)
+      true
+    end
+
+    private
+
+    def hash
+      Digest::MD5.hexdigest @key
+    end
+
+    def store
+      APICache.store
+    end
   end
 end

data/lib/api_cache/memory_store.rb

@@ -1,33 +1,35 @@
-class APICache::MemoryStore < APICache::AbstractStore
-  def initialize
-    APICache.logger.log "Using memory store"
-    @cache = {}
-    true
-  end
+class APICache
+  class MemoryStore < APICache::AbstractStore
+    def initialize
+      APICache.logger.debug "Using memory store"
+      @cache = {}
+      true
+    end
 
-  def set(key, value)
-    APICache.logger.log("cache: set (#{key})")
-    @cache[key] = [Time.now, value]
-    true
-  end
+    def set(key, value)
+      APICache.logger.debug("cache: set (#{key})")
+      @cache[key] = [Time.now, value]
+      true
+    end
 
-  def get(key)
-    data = @cache[key][1]
-    APICache.logger.log("cache: #{data.nil? ? "miss" : "hit"} (#{key})")
-    data
-  end
+    def get(key)
+      data = @cache[key][1]
+      APICache.logger.debug("cache: #{data.nil? ? "miss" : "hit"} (#{key})")
+      data
+    end
 
-  def exists?(key)
-    !@cache[key].nil?
-  end
-  
-  def expired?(key, timeout)
-    Time.now - created(key) > timeout
-  end
+    def exists?(key)
+      !@cache[key].nil?
+    end
+
+    def expired?(key, timeout)
+      Time.now - created(key) > timeout
+    end
 
-  private
+    private
 
-  def created(key)
-    @cache[key][0]
+    def created(key)
+      @cache[key][0]
+    end
   end
 end

data/lib/api_cache/moneta_store.rb

@@ -0,0 +1,29 @@
+class APICache
+  class MonetaStore < APICache::AbstractStore
+    def initialize(store)
+      @moneta = store
+    end
+
+    # Set value. Returns true if success.
+    def set(key, value)
+      @moneta[key] = value
+      @moneta["#{key}_created_at"] = Time.now
+      true
+    end
+
+    # Get value.
+    def get(key)
+      @moneta[key]
+    end
+
+    # Does a given key exist in the cache?
+    def exists?(key)
+      @moneta.key?(key)
+    end
+
+    # Has a given time passed since the key was set?
+    def expired?(key, timeout)
+      Time.now - @moneta["#{key}_created_at"] > timeout
+    end
+  end
+end

data/spec/api_cache_spec.rb

@@ -3,70 +3,99 @@ require File.dirname(__FILE__) + '/spec_helper'
 describe APICache do
   before :each do
     @key = 'random_key'
-    @encoded_key = "1520171c64bfb71a95c97d310eea3492"
-    @data = 'some bit of data'
+    @cache_data = 'data from the cache'
+    @api_data = 'data from the api'
   end
   
-  describe "get method" do
-    
-    it "should MD5 encode the cache key" do
-      APICache::Cache.new(APICache::MemoryStore).encode(@key).should == @encoded_key
+  describe "store=" do
+    before :each do
+      APICache.store = nil
+    end
+
+    it "should use APICache::MemoryStore by default" do
+      APICache.store.should be_kind_of(APICache::MemoryStore)
+    end
+
+    it "should allow instances of APICache::AbstractStore to be passed" do
+      APICache.store = APICache::MemoryStore.new
+      APICache.store.should be_kind_of(APICache::MemoryStore)
+    end
+
+    it "should allow moneta instances to be passed" do
+      require 'moneta'
+      require 'moneta/memory'
+      APICache.store = Moneta::Memory.new
+      APICache.store.should be_kind_of(APICache::MonetaStore)
     end
-    it "should encode the cache key before calling the store" do
-      APICache.cache.should_receive(:state).and_return(:current)
-      APICache.cache.store.should_receive(:get).with(@encoded_key).and_return(@data)
-      APICache.cache.should_receive(:encode).with(@key).and_return(@encoded_key)
-      APICache.get(@key).should != @data
+
+    it "should raise an exception if anything else is passed" do
+      lambda {
+        APICache.store = Class
+      }.should raise_error(ArgumentError, 'Please supply an instance of either a moneta store or a subclass of APICache::AbstractStore')
+    end
+
+    after :all do
+      APICache.store = nil
+    end
+  end
+
+  describe "get method" do
+    before :each do
+      @api = mock(APICache::API, :get => @api_data)
+      @cache = mock(APICache::Cache, :get => @cache_data, :set => true)
+      
+      APICache::API.stub!(:new).and_return(@api)
+      APICache::Cache.stub!(:new).and_return(@cache)
     end
     
     it "should fetch data from the cache if the state is :current" do
-      APICache.cache.should_receive(:state).and_return(:current)
-      APICache.cache.should_receive(:get).and_return(@data)
+      @cache.stub!(:state).and_return(:current)
       
-      APICache.get(@key).should == @data
+      APICache.get(@key).should == @cache_data
     end
     
-    it "should make new request to API if the state is :refetch" do
-      APICache.cache.should_receive(:state).and_return(:refetch)
-      APICache.api.should_receive(:get).with(@key, 5).and_return(@data)
+    it "should make new request to API if the state is :refetch and store result in cache" do
+      @cache.stub!(:state).and_return(:refetch)
+      @cache.should_receive(:set).with(@api_data)
       
-      APICache.get(@key).should == @data
+      APICache.get(@key).should == @api_data
     end
     
-    it "should return the cached value if the api cannot fetch data and state is :refetch" do
-      APICache.cache.should_receive(:state).and_return(:refetch)
-      APICache.api.should_receive(:get).with(@key, 5).and_raise(APICache::CannotFetch)
-      APICache.cache.should_receive(:get).and_return(@data)
+    it "should return the cached value if the state is :refetch but the api is not accessible" do
+      @cache.stub!(:state).and_return(:refetch)
+      @api.should_receive(:get).with.and_raise(APICache::CannotFetch)
       
-      APICache.get(@key).should == @data
+      APICache.get(@key).should == @cache_data
     end
     
     it "should make new request to API if the state is :invalid" do
-      APICache.cache.should_receive(:state).and_return(:invalid)
-      APICache.api.should_receive(:get).with(@key, 5).and_return(@data)
+      @cache.stub!(:state).and_return(:invalid)
       
-      APICache.get(@key).should == @data
+      APICache.get(@key).should == @api_data
     end
     
-    it "should raise an exception if the api cannot fetch data and state is :invalid" do
-      APICache.cache.should_receive(:state).and_return(:invalid)
-      APICache.api.should_receive(:get).with(@key, 5).and_raise(APICache::CannotFetch)
+    it "should raise CannotFetch if the api cannot fetch data and the cache state is :invalid" do
+      @cache.stub!(:state).and_return(:invalid)
+      @api.should_receive(:get).with.and_raise(APICache::CannotFetch)
       
-      lambda { APICache.get(@key).should }.should raise_error(APICache::NotAvailableError)
+      lambda {
+        APICache.get(@key).should
+      }.should raise_error(APICache::CannotFetch)
     end
     
     it "should make new request to API if the state is :missing" do
-      APICache.cache.should_receive(:state).and_return(:missing)
-      APICache.api.should_receive(:get).with(@key, 5).and_return(@data)
+      @cache.stub!(:state).and_return(:missing)
       
-      APICache.get(@key).should == @data
+      APICache.get(@key).should == @api_data
     end
     
-    it "should raise an exception if the api cannot fetch data and state is :missing" do
-      APICache.cache.should_receive(:state).and_return(:missing)
-      APICache.api.should_receive(:get).with(@key, 5).and_raise(APICache::CannotFetch)
+    it "should raise an exception if the api cannot fetch data and the cache state is :missing" do
+      @cache.stub!(:state).and_return(:missing)
+      @api.should_receive(:get).with.and_raise(APICache::CannotFetch)
       
-      lambda { APICache.get(@key).should }.should raise_error(APICache::NotAvailableError)
+      lambda {
+        APICache.get(@key).should
+      }.should raise_error(APICache::CannotFetch)
     end
   end
 end

data/spec/api_spec.rb

@@ -1,10 +1,65 @@
 require File.dirname(__FILE__) + '/spec_helper'
 
 describe APICache::API do
-  it "should handle redirecting get requests" do
-    api = APICache::API.new
+  before :each do
+    FakeWeb.register_uri(:get, "http://www.google.com/", :body => "Google")
+    FakeWeb.register_uri(:get, "http://froogle.google.com/", :status => 302, :location => "http://products.google.com")
+    FakeWeb.register_uri(:get, "http://products.google.com/", :body => "Google Product Search")
+
+    @options = {
+      :period => 1,
+      :timeout => 5
+    }
+
+    # Reset the store otherwise get queried too recently erros
+    APICache.store = nil
+  end
+
+  it "should not be queryable for :period seconds after a request" do
+    api = APICache::API.new('http://www.google.com/', @options)
+
+    api.get
     lambda {
-      api.get('http://froogle.google.com', 5)
-    }.should_not raise_error(APICache::CannotFetch)
+      api.get
+    }.should raise_error(APICache::CannotFetch, "Cannot fetch http://www.google.com/: queried too recently")
+
+    sleep 1
+    lambda {
+      api.get
+    }.should_not raise_error
+  end
+
+  describe "without a block - key is the url" do
+
+    it "should return body of a http GET against the key" do
+      api = APICache::API.new('http://www.google.com/', @options)
+      api.get.should =~ /Google/
+    end
+
+    it "should handle redirecting get requests" do
+      api = APICache::API.new('http://froogle.google.com/', @options)
+      api.get.should =~ /Google Product Search/
+    end
+
+  end
+
+  describe "with a block" do
+
+    it "should return the return value of the block" do
+      api = APICache::API.new('http://www.google.com/', @options) do
+        42
+      end
+      api.get.should == 42
+    end
+
+    it "should return the raised exception if the block raises one" do
+      api = APICache::API.new('foobar', @options) do
+        raise RuntimeError, 'foo'
+      end
+      lambda {
+        api.get
+      }.should raise_error(StandardError, 'foo')
+    end
+
   end
-end
+end

data/spec/cache_spec.rb

@@ -0,0 +1,36 @@
+require File.dirname(__FILE__) + '/spec_helper'
+
+describe APICache::Cache do
+  before :each do
+    @options = {
+      :cache => 1,    # After this time fetch new data
+      :valid => 2   # Maximum time to use old data
+    }
+  end
+
+  it "should set and get" do
+    cache = APICache::Cache.new('flubble', @options)
+
+    cache.set('Hello world')
+    cache.get.should == 'Hello world'
+  end
+
+  it "should md5 encode the provided key" do
+    cache = APICache::Cache.new('test_md5', @options)
+    APICache.store.should_receive(:set).
+      with('9050bddcf415f2d0518804e551c1be98', 'md5ing?')
+    cache.set('md5ing?')
+  end
+
+  it "should report correct invalid states" do
+    cache = APICache::Cache.new('foo', @options)
+
+    cache.state.should == :missing
+    cache.set('foo')
+    cache.state.should == :current
+    sleep 1
+    cache.state.should == :refetch
+    sleep 1
+    cache.state.should == :invalid
+  end
+end

data/spec/integration_spec.rb

@@ -0,0 +1,73 @@
+require File.dirname(__FILE__) + '/spec_helper'
+
+describe "api_cache" do
+  before :each do
+    FakeWeb.register_uri(:get, "http://www.google.com/", :body => "Google")
+
+    APICache.store = nil
+  end
+
+  it "should work when url supplied" do
+    APICache.get('http://www.google.com/').should =~ /Google/
+  end
+
+  it "should work when block supplied" do
+    APICache.get('foobar') do
+      42
+    end.should == 42
+  end
+
+  it "should raise error raised in block unless key available in cache" do
+    lambda {
+      APICache.get('foo') do
+        raise RuntimeError, 'foo'
+      end
+    }.should raise_error(RuntimeError, 'foo')
+
+    APICache.get('foo', :period => 0) do
+      'bar'
+    end
+
+    lambda {
+      APICache.get('foo') do
+        raise RuntimeError, 'foo'
+      end
+    }.should_not raise_error
+  end
+
+  it "should raise APICache::TimeoutError if the API call times out unless data available in cache" do
+    lambda {
+      APICache.get('foo', :timeout => 1) do
+        sleep 1.1
+      end
+    }.should raise_error APICache::TimeoutError, 'Timed out when calling API (timeout 1s)'
+
+    APICache.get('foo', :period => 0) do
+      'bar'
+    end
+
+    lambda {
+      APICache.get('foo', :timeout => 1) do
+        sleep 1.1
+      end
+    }.should_not raise_error
+  end
+
+  it "should return a default value rather than raising an exception if :fail passed" do
+    APICache.get('foo', :fail => "bar") do
+      raise 'foo'
+    end.should == 'bar'
+  end
+
+  it "should accept a proc to fail" do
+    APICache.get('foo', :fail => lambda { "bar" }) do
+      raise 'foo'
+    end.should == 'bar'
+  end
+
+  it "should accept nil values for :fail" do
+    APICache.get('foo', :fail => nil) do
+      raise 'foo'
+    end.should == nil
+  end
+end

data/spec/monteta_store_spec.rb

@@ -0,0 +1,30 @@
+require File.dirname(__FILE__) + '/spec_helper'
+
+require 'moneta/memcache'
+
+describe APICache::MonetaStore do
+  before :each do
+    @moneta = Moneta::Memcache.new(:server => "localhost")
+    @moneta.delete('foo')
+    @store = APICache::MonetaStore.new(@moneta)
+  end
+
+  it "should set and get" do
+    @store.set("key", "value")
+    @store.get("key").should == "value"
+  end
+
+  it "should allow checking whether a key exists" do
+    @store.exists?('foo').should be_false
+    @store.set('foo', 'bar')
+    @store.exists?('foo').should be_true
+  end
+
+  it "should allow checking whether a given amount of time has passed since the key was set" do
+    @store.expired?('foo', 1).should be_false
+    @store.set('foo', 'bar')
+    @store.expired?('foo', 1).should be_false
+    sleep 1
+    @store.expired?('foo', 1).should be_true
+  end
+end

data/spec/spec_helper.rb

@@ -1,7 +1,7 @@
-$TESTING=true
 $:.push File.join(File.dirname(__FILE__), '..', 'lib')
 require "rubygems"
 require "api_cache"
 require "spec"
+require "fakeweb"
 
-APICache.start(APICache::MemoryStore)
+APICache.logger.level = Logger::FATAL

metadata

@@ -1,7 +1,7 @@
 --- !ruby/object:Gem::Specification 
 name: api_cache
 version: !ruby/object:Gem::Version 
-  version: 0.1.2
+  version: 0.2.0
 platform: ruby
 authors: 
 - Martyn Loughran
@@ -9,11 +9,30 @@ autorequire:
 bindir: bin
 cert_chain: []
 
-date: 2009-03-30 00:00:00 +02:00
+date: 2009-08-21 00:00:00 +02:00
 default_executable: 
-dependencies: []
-
-description: Library to handle caching external API calls
+dependencies: 
+- !ruby/object:Gem::Dependency 
+  name: rspec
+  type: :development
+  version_requirement: 
+  version_requirements: !ruby/object:Gem::Requirement 
+    requirements: 
+    - - ">="
+      - !ruby/object:Gem::Version 
+        version: "0"
+    version: 
+- !ruby/object:Gem::Dependency 
+  name: fakeweb
+  type: :development
+  version_requirement: 
+  version_requirements: !ruby/object:Gem::Requirement 
+    requirements: 
+    - - ">="
+      - !ruby/object:Gem::Version 
+        version: "0"
+    version: 
+description: APICache allows any API client library to be easily wrapped with a robust caching layer. It supports caching (obviously), serving stale data and limits on the number of API calls. It's also got a handy syntax if all you want to do is cache a bothersome url.
 email: me@mloughran.com
 executables: []
 
@@ -22,20 +41,22 @@ extensions: []
 extra_rdoc_files: []
 
 files: 
-- README.markdown
+- README.rdoc
 - VERSION.yml
 - lib/api_cache/abstract_store.rb
 - lib/api_cache/api.rb
 - lib/api_cache/cache.rb
-- lib/api_cache/logger.rb
-- lib/api_cache/memcache_store.rb
 - lib/api_cache/memory_store.rb
+- lib/api_cache/moneta_store.rb
 - lib/api_cache.rb
 - spec/api_cache_spec.rb
 - spec/api_spec.rb
+- spec/cache_spec.rb
+- spec/integration_spec.rb
+- spec/monteta_store_spec.rb
 - spec/spec_helper.rb
 has_rdoc: true
-homepage: http://github.com/mloughran/api_cache
+homepage: http://mloughran.github.com/api_cache/
 licenses: []
 
 post_install_message: 
@@ -61,7 +82,7 @@ requirements: []
 rubyforge_project: 
 rubygems_version: 1.3.5
 signing_key: 
-specification_version: 2
-summary: Library to handle caching external API calls
+specification_version: 3
+summary: API Cache allows advanced caching of APIs
 test_files: []
 

data/README.markdown

@@ -1,80 +0,0 @@
-APICache (aka api_cache)
-========================
-
-For the impatient
------------------
-
-    # Install
-    sudo gem install mloughran-api_cache -s http://gems.github.com
-    
-    # Require
-    require 'rubygems'
-    gem 'mloughran-api_cache'
-    require 'api_cache'
-    
-    # Configure
-    APICache.start(APICache::MemoryStore)
-    
-    # Use
-    APICache.get("http://twitter.com/statuses/public_timeline.rss")
-
-For everyone else
------------------
-
-You want to use the Twitter API but you don't want to die? I have the solution to API caching:
-
-    APICache.get("http://twitter.com/statuses/public_timeline.rss")
-
-You get the following functionality for free:
-
-* New data every 10 minutes
-* If the twitter API dies then keep using the last data received for a day. Then assume it's invalid and announce that Twitter has FAILED (optional).
-* Don't hit the rate limit (70 requests per 60 minutes)
-
-So what exactly does `APICache` do? Given cached data less than 10 minutes old, it returns that. Otherwise, assuming it didn't try to request the URL within the last minute (to avoid the rate limit), it makes a get request to the Twitter API. If the Twitter API timeouts or doesn't return a 2xx code (very likely) we're still fine: it just returns the last data fetched (as long as it's less than a day old). In the exceptional case that all is lost and no data can be returned, it raises an `APICache::NotAvailableError` exception. You're responsible for catching this exception and complaining bitterly to the internet.
-
-All very simple. What if you need to do something more complicated? Say you need authentication or the silly API you're using doesn't follow a nice convention of returning 2xx for success. Then you need a block:
-
-    APICache.get('twitter_replies', :cache => 3600) do
-      Net::HTTP.start('twitter.com') do |http|
-        req = Net::HTTP::Get.new('/statuses/replies.xml')
-        req.basic_auth 'username', 'password'
-        response = http.request(req)
-        case response
-        when Net::HTTPSuccess
-          # 2xx response code
-          response.body
-        else
-          raise APICache::Invalid
-        end
-      end
-    end
-
-All the caching is still handled for you. If you supply a block then the first argument to `APICache.get` is assumed to be a unique key rather than a URL. Throwing `APICache::Invalid` signals to `APICache` that the request was not successful.
-
-You can send any of the following options to `APICache.get(url, options = {}, &block)`. These are the default values (times are all in seconds):
-
-    {
-      :cache => 600,    # 10 minutes  After this time fetch new data
-      :valid => 86400,  # 1 day       Maximum time to use old data
-                        #             :forever is a valid option
-      :period => 60,    # 1 minute    Maximum frequency to call API
-      :timeout => 5     # 5 seconds   API response timeout
-    }
-
-Before using the APICache you need to initialize the caches. In merb, for example, put this in your `init.rb`:
-
-    APICache.start
-
-Currently there are two stores available: `MemcacheStore` and `MemoryStore`. `MemcacheStore` is the default but if you'd like to use `MemoryStore`, or another store - see `AbstractStore`, just supply it to the start method:
-
-    APICache.start(APICache::MemoryStore)
-
-I suppose you'll want to get your hands on this magic! Just take a look at the instructions above for the impatient. Well done for reading this first!
-
-Please send feedback to me [at] mloughran [dot] com if you think of any other functionality that would be handy.
-
-Copyright
-=========
-
-Copyright (c) 2008 Martyn Loughran. See LICENSE for details.

data/lib/api_cache/logger.rb

@@ -1,9 +0,0 @@
-class APICache::Logger
-  def initialize
-    
-  end
-  
-  def log(message)
-    # puts "APICache: #{message}"
-  end
-end

data/lib/api_cache/memcache_store.rb

@@ -1,54 +0,0 @@
-require 'memcache'
-
-class APICache::MemcacheStore < APICache::AbstractStore
-  class NotReady < Exception #:nodoc:
-    def initialize
-      super("Memcache server is not ready")
-    end
-  end
-
-  class NotDefined < Exception #:nodoc:
-    def initialize
-      super("Memcache is not defined")
-    end
-  end
-  
-  def initialize
-    APICache.logger.log "Using memcached store"
-    namespace = 'api_cache'
-    host = '127.0.0.1:11211'
-    @memcache = MemCache.new(host, {:namespace => namespace})
-    raise NotReady unless @memcache.active?
-    true
-  # rescue NameError
-  #   raise NotDefined
-  end
-  
-  def set(key, data)
-    @memcache.set(key, data)
-    @memcache.set("#{key}_created_at", Time.now)
-    APICache.logger.log("cache: set (#{key})")
-    true
-  end
-  
-  def get(key)
-    data = @memcache.get(key)
-    APICache.logger.log("cache: #{data.nil? ? "miss" : "hit"} (#{key})")
-    data
-  end
-  
-  def exists?(key)
-    # TODO: inefficient - is there a better way?
-    !@memcache.get(key).nil?
-  end
-  
-  def expired?(key, timeout)
-    Time.now - created(key) > timeout
-  end
-  
-  private
-
-  def created(key)
-    @memcache.get("#{key}_created_at")
-  end
-end