readthis 0.8.1 → 1.2.1

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA1:
-  metadata.gz: ea629e7d22f58baa04359d6fabaee9116caf4e5d
-  data.tar.gz: ff5720742c337be8b178ca840064b3469766dcda
+  metadata.gz: 2c3b9c325211a544d3cdffbbb4356c7c392a5c04
+  data.tar.gz: ecd6d0982a96bcff0f9bfafff60b6d758ab5ee55
 SHA512:
-  metadata.gz: b75d0eb316bb68975693ab64044cc3c30159b9511dde446b504e1075b5c6aa915bc3782290334001de3b921173714343c86110778bc41ec90f67dc84b9ef0704
-  data.tar.gz: 765fae41b2f8d511cd99269e17ed31eacbb968239e97be78422a4b20793ae3325feefc2f19d411b6d0a17722a7b06bbba78230fe49e12c76e01edabb9fc93b43
+  metadata.gz: 92323a5bfb41f8a29ea0b3082ff500078af9944d85fd756a1147b2cc1a7899e7fa4900c086463bfaa775608cf3d7c86137b4e54084f72ce1a2110b3f14d55c2b
+  data.tar.gz: f2d19489b82236baee8be73033b07f3a375585195fc251b17c5ce9f15972911e696e038b282a949a513c8afda4465706f2a581b90c99529da7255e364fe9517b

data/README.md

@@ -5,9 +5,10 @@
 
 # Readthis
 
-Readthis is a drop in replacement for any ActiveSupport compliant cache. It
-emphasizes performance and simplicity and takes some cues from Dalli the popular
-Memcache client.
+Readthis is a Redis backed cache client for Ruby. It is a drop in replacement
+for any `ActiveSupport` compliant cache and can also be used for [session
+storage](#session-storage).  Above all Readthis emphasizes performance,
+simplicity, and explicitness.
 
 For new projects there isn't any reason to stick with Memcached. Redis is as
 fast, if not faster in many scenarios, and is far more likely to be used
@@ -52,7 +53,11 @@ cache = Readthis::Cache.new(
 )
 ```
 
+You can also specify `host`, `port`, `db` or any other valid Redis options. For
+more details about connection options see in [redis gem documentation][redisrb]
+
 [store]: http://api.rubyonrails.org/classes/ActiveSupport/Cache/Store.html
+[redisrb]: https://github.com/redis/redis-rb#getting-started
 
 ### Instances & Databases
 
@@ -61,24 +66,29 @@ instances have numerous benefits like: more predictable performance, avoiding
 expires in favor of LRU, and tuning the persistence mechanism. See [Optimizing
 Redis Usage for Caching][optimizing-usage] for more details.
 
-[optimizing-usage]: http://sorentwo.com/2015/07/27/optimizing-redis-usage-for-caching.html
-
-At the very least you'll want to use a specific database for caching. In the
+At the very least, you'll want to use a specific database for caching. In the
 event the database needs to be purged you can do so with a single `clear`
 command, rather than finding all keys in a namespace and deleting them.
 Appending a number between 0 and 15 will specify the redis database, which
-defaults to 0. For example, using database 2:
+defaults to `0`. For example, using database `2`:
 
 ```bash
 REDIS_URL=redis://localhost:6379/2
 ```
 
+[optimizing-usage]: http://sorentwo.com/2015/07/27/optimizing-redis-usage-for-caching.html
+
 ### Expiration
 
 Be sure to use an integer value when setting expiration time. The default
 representation of `ActiveSupport::Duration` values won't work when setting
 expiration time, which will cause all keys to have `-1` as the TTL. Expiration
-values are always cast as an integer on write.
+values are always cast as an integer on write. For example:
+
+```ruby
+Readthis::Cache.new(expires_in: 1.week) # don't do this
+Readthis::Cache.new(expires_in: 1.week.to_i) # do this
+```
 
 ### Compression
 
@@ -96,23 +106,64 @@ config.cache_store = :readthis_store, {
 }
 ```
 
-### Marshalling
+### Serializing
+
+Readthis uses Ruby's `Marshal` module for serializing all values by default.
+This isn't always the fastest option, and depending on your use case it may be
+desirable to use a faster but less flexible serializer.
+
+By default Readthis knows about 3 different serializers:
 
-Readthis uses Ruby's `Marshal` module for dumping and loading all values by
-default. This isn't always the fastest option, and depending on your use case it
-may be desirable to use a faster but less flexible marshaller.
+* Marshal
+* JSON
+* Passthrough
 
-Use Oj for JSON marshalling, extremely fast, but supports limited types:
+If all cached data can safely be represented as a string then use the
+pass-through serializer:
+
+```ruby
+Readthis::Cache.new(marshal: Readthis::Passthrough)
+```
+
+You can introduce up to four additional serializers by configuring `serializers`
+on the Readthis module. For example, if you wanted to use the extremely fast Oj
+library for JSON serialization:
 
 ```ruby
+Readthis.serializers << Oj
+
+# Freeze the serializers to ensure they aren't changed at runtime.
+Readthis.serializers.freeze!
+
 Readthis::Cache.new(marshal: Oj)
 ```
 
-If all cached data can safely be represented as a string then use the
-pass-through marshaller:
+Be aware that the *order in which you add serializers matters*. Serializers are
+sticky and a flag is stored with each cached value. If you subsequently go to
+deserialize values and haven't configured the same serializers in the same order
+your application will raise errors.
+
+## Fault Tolerance
+
+In some situations it is desirable to keep serving requests from disk or the
+database if Redis crashes. This can be achieved with connection fault tolerance
+by enabling it at the top level:
 
 ```ruby
-Readthis::Cache.new(marshal: Readthis::Passthrough)
+Readthis.fault_tolerant = true
+```
+
+The default value is `false`, because while it may work for `fetch` operations,
+it isn't compatible with other state-based commands like `increment`.
+
+## Running Arbitrary Redis Commands
+
+Readthis provides access to the underlying Redis connection pool, allowing you
+to run arbitrary commands directly through the cache instance. For example, if
+you wanted to expire a key manually using an instance of `Rails.cache`:
+
+```ruby
+Rails.cache.pool.with { |client| client.expire('foo-key', 60) }
 ```
 
 ## Differences From ActiveSupport::Cache
@@ -122,11 +173,34 @@ Readthis supports all of standard cache methods except for the following:
 * `cleanup` - Redis does this with TTL or LRU already.
 * `delete_matched` - You really don't want to perform key matching operations in
   Redis. They are linear time and only support basic globbing.
+* `mute` and `silence!` - You can subscribe to the events `/cache*+active_support/` with `ActiveSupport::Notifications` to [log cache calls manually][notifications].
+
+[notifications]: https://github.com/sorentwo/readthis/issues/22#issuecomment-142595938
 
 Like other `ActiveSupport::Cache` implementations it is possible to cache `nil`
 as a value. However, the fetch methods treat `nil` values as a cache miss and
 re-generate/re-cache the value. Caching `nil` isn't recommended.
 
+## Session Storage
+
+By using [ActionDispatch::Session::CacheStore][cache-store] it's possible to
+reuse `:readthis_store` or specify a new Readthis cache store for storing
+sessions.
+
+```ruby
+Rails.application.config.session_store :cache_store
+```
+
+To specify a separate Readthis instance you can use the `:cache` option:
+
+```ruby
+Rails.application.config.session_store :cache_store,
+  cache: Readthis::Cache.new,
+  expire_after: 2.weeks.to_i
+```
+
+[cache-store]: http://api.rubyonrails.org/classes/ActionDispatch/Session/CacheStore.html
+
 ## Contributing
 
 1. Fork it

data/lib/active_support/cache/readthis_store.rb

@@ -2,6 +2,6 @@ require 'readthis'
 
 module ActiveSupport
   module Cache
-    ReadthisStore ||= Readthis::Cache
+    ReadthisStore ||= Readthis::Cache # rubocop:disable Style/ConstantName
   end
 end

data/lib/readthis/cache.rb

@@ -1,24 +1,19 @@
 require 'readthis/entity'
 require 'readthis/expanders'
-require 'readthis/notifications'
 require 'readthis/passthrough'
 require 'redis'
 require 'connection_pool'
 
 module Readthis
   class Cache
-    attr_reader :entity, :expires_in, :namespace, :options, :pool
+    attr_reader :entity, :notifications, :options, :pool
 
     # Provide a class level lookup of the proper notifications module.
     # Instrumention is expected to occur within applications that have
     # ActiveSupport::Notifications available, but needs to work even when it
     # isn't.
     def self.notifications
-      if defined?(ActiveSupport::Notifications)
-        ActiveSupport::Notifications
-      else
-        Readthis::Notifications
-      end
+      ActiveSupport::Notifications if defined?(ActiveSupport::Notifications)
     end
 
     # Creates a new Readthis::Cache object with the given options.
@@ -39,9 +34,7 @@ module Readthis
     #   Readthis::Cache.new(compress: true, compression_threshold: 2048)
     #
     def initialize(options = {})
-      @options    = options
-      @expires_in = options.fetch(:expires_in, nil)
-      @namespace  = options.fetch(:namespace, nil)
+      @options = options
 
       @entity = Readthis::Entity.new(
         marshal:   options.fetch(:marshal, Marshal),
@@ -145,6 +138,7 @@ module Readthis
     #   cache.fetch('today', force: true) # => nil
     #
     def fetch(key, options = {})
+      options ||= {}
       value = read(key, options) unless options[:force]
 
       if value.nil? && block_given?
@@ -171,7 +165,7 @@ module Readthis
     #   cache.increment('counter', 2) # => 3
     #
     def increment(key, amount = 1, options = {})
-      invoke(:incremenet, key) do |store|
+      invoke(:increment, key) do |_store|
         alter(key, amount, options)
       end
     end
@@ -192,7 +186,7 @@ module Readthis
     #   cache.decrement('counter', 2) # => 17
     #
     def decrement(key, amount = 1, options = {})
-      invoke(:decrement, key) do |store|
+      invoke(:decrement, key) do |_store|
         alter(key, amount * -1, options)
       end
     end
@@ -218,7 +212,7 @@ module Readthis
       return {} if keys.empty?
 
       invoke(:read_multi, keys) do |store|
-        values = store.mget(mapping).map { |value| entity.load(value) }
+        values = store.mget(*mapping).map { |value| entity.load(value) }
 
         keys.zip(values).to_h
       end
@@ -271,13 +265,13 @@ module Readthis
       extracted = extract_options!(keys)
       missing   = {}
 
-      invoke(:fetch_multi, keys) do |store|
+      invoke(:fetch_multi, keys) do |_store|
         results.each do |key, value|
-          if value.nil?
-            value = yield(key)
-            missing[key] = value
-            results[key] = value
-          end
+          next unless value.nil?
+
+          value = yield(key)
+          missing[key] = value
+          results[key] = value
         end
       end
 
@@ -310,7 +304,7 @@ module Readthis
     # @example
     #
     #   cache.clear #=> 'OK'
-    def clear(options = {})
+    def clear(_options = nil)
       invoke(:clear, '*', &:flushdb)
     end
 
@@ -318,11 +312,12 @@ module Readthis
 
     def write_entity(key, value, store, options)
       namespaced = namespaced_key(key, options)
+      dumped = entity.dump(value, options)
 
       if expiration = options[:expires_in]
-        store.setex(namespaced, expiration.to_i, entity.dump(value))
+        store.setex(namespaced, expiration.to_i, dumped)
       else
-        store.set(namespaced, entity.dump(value))
+        store.set(namespaced, dumped)
       end
     end
 
@@ -335,17 +330,23 @@ module Readthis
       delta
     end
 
-    def instrument(operation, key)
-      name    = "cache_#{operation}.active_support"
-      payload = { key: key }
+    def instrument(name, key)
+      if self.class.notifications
+        name    = "cache_#{name}.active_support"
+        payload = { key: key, name: name }
 
-      self.class.notifications.instrument(name, key) { yield(payload) }
+        self.class.notifications.instrument(name, payload) { yield(payload) }
+      else
+        yield
+      end
     end
 
     def invoke(operation, key, &block)
       instrument(operation, key) do
         pool.with(&block)
       end
+    rescue Redis::BaseError => error
+      raise error unless Readthis.fault_tolerant?
     end
 
     def extract_options!(array)
@@ -353,10 +354,7 @@ module Readthis
     end
 
     def merged_options(options)
-      options = options || {}
-      options[:namespace]  ||= namespace
-      options[:expires_in] ||= expires_in
-      options
+      @options.merge(options || {})
     end
 
     def pool_options(options)

data/lib/readthis/entity.rb

@@ -2,51 +2,143 @@ require 'zlib'
 
 module Readthis
   class Entity
-    DEFAULT_THRESHOLD = 8 * 1024
-    MAGIC_BYTES = [120, 156].freeze
+    DEFAULT_OPTIONS = {
+      compress:  false,
+      marshal:   Marshal,
+      threshold: 8 * 1024
+    }.freeze
 
-    attr_reader :marshal, :compression, :threshold
+    COMPRESSED_FLAG = 0x8
 
+    # Creates a Readthis::Entity with default options. Each option can be
+    # overridden later when entities are being dumped.
+    #
+    # Options are sticky, meaning that whatever is used when dumping will
+    # automatically be used again when loading, regardless of how current
+    # options are set.
+    #
+    # @option [Boolean] :compress (false) Enable or disable automatic compression
+    # @option [Module]  :marshal (Marshal) Any module that responds to `dump` and `load`
+    # @option [Number]  :threshold (8k) The size a string must be for compression
+    #
     def initialize(options = {})
-      @marshal     = options.fetch(:marshal, Marshal)
-      @compression = options.fetch(:compress, false)
-      @threshold   = options.fetch(:threshold, DEFAULT_THRESHOLD)
+      @options = DEFAULT_OPTIONS.merge(options)
     end
 
-    def dump(value)
-      if compress?(value)
-        compress(value)
-      else
-        marshal.dump(value)
-      end
+    # Output a value prepared for cache storage. Passed options will override
+    # whatever has been specified for the instance.
+    #
+    # @param  [String] String to dump
+    # @option [Boolean] :compress Enable or disable automatic compression
+    # @option [Module]  :marshal Any module that responds to `dump` and `load`
+    # @option [Number]  :threshold The size a string must be for compression
+    # @return [String] The prepared, possibly compressed, string
+    #
+    # @example Dumping a value using defaults
+    #
+    #   entity.dump(string)
+    #
+    # @example Dumping a value with overrides
+    #
+    #   entity.dump(string, compress: false, marshal: JSON)
+    #
+    def dump(value, options = {})
+      compress  = with_fallback(options, :compress)
+      marshal   = with_fallback(options, :marshal)
+      threshold = with_fallback(options, :threshold)
+
+      dumped = deflate(marshal.dump(value), compress, threshold)
+
+      compose(dumped, marshal, compress)
     end
 
-    def load(value)
-      if compressed?(value)
-        decompress(value)
-      else
-        marshal.load(value)
-      end
-    rescue TypeError, Zlib::Error
-      value
+    # Parse a dumped value using the embedded options.
+    #
+    # @param  [String] Option embedded string to load
+    # @return [String] The original dumped string, restored
+    #
+    # @example
+    #
+    #   entity.load(dumped)
+    #
+    def load(string)
+      marshal, compress, value = decompose(string)
+
+      marshal.load(inflate(value, compress))
+    rescue TypeError, NoMethodError
+      string
     end
 
-    def compress(value)
-      Zlib::Deflate.deflate(marshal.dump(value))
+    # Composes a single byte comprised of the chosen serializer and compression
+    # options. The byte is formatted as:
+    #
+    # | 0000 | 0 | 000 |
+    #
+    # Where there are four unused bits, 1 compression bit, and 3 bits for the
+    # serializer. This allows up to 8 different serializers for marshaling.
+    #
+    # @param  [String] String to prefix with flags
+    # @param  [Module] The marshal module to be used
+    # @param  [Boolean] Flag determining whether the value is compressed
+    # @return [String] The original string with a single byte prefixed
+    #
+    # @example Compose an option embedded string
+    #
+    #   entity.compose(string, Marshal, false) => 0x1  + string
+    #   entity.compose(string, JSON, true)     => 0x10 + string
+    #
+    def compose(value, marshal, compress)
+      flags  = serializers.assoc(marshal)
+      flags |= COMPRESSED_FLAG if compress
+
+      value.prepend([flags].pack('C'))
     end
 
-    def decompress(value)
-      marshal.load(Zlib::Inflate.inflate(value))
+    # Decompose an option embedded string into marshal, compression and value.
+    #
+    # @param  [String] Option embedded string to
+    # @return [Array<Module, Boolean, String>] An array comprised of the
+    #   marshal, compression flag, and the base string.
+    #
+    def decompose(string)
+      flags = string[0].unpack('C').first
+
+      if flags < 16
+        marshal  = serializers.rassoc(flags)
+        compress = (flags & COMPRESSED_FLAG) != 0
+
+        [marshal, compress, string[1..-1]]
+      else
+        [@options[:marshal], @options[:compress], string]
+      end
     end
 
     private
 
-    def compress?(value)
-      compression && value.bytesize >= threshold
+    def deflate(value, compress, threshold)
+      if compress && value.bytesize >= threshold
+        Zlib::Deflate.deflate(value)
+      else
+        value
+      end
+    end
+
+    def inflate(value, decompress)
+      if decompress
+        Zlib::Inflate.inflate(value)
+      else
+        value
+      end
+    rescue Zlib::Error
+      value
+    end
+
+    def serializers
+      Readthis.serializers
     end
 
-    def compressed?(value)
-      compression && value[0, 2].unpack('CC') == MAGIC_BYTES
+    def with_fallback(options, key)
+      options.key?(key) ? options[key] : @options[key]
     end
   end
 end

data/lib/readthis/errors.rb

@@ -0,0 +1,7 @@
+module Readthis
+  ReadthisError = Class.new(StandardError)
+
+  SerializersFrozenError = Class.new(ReadthisError)
+  SerializersLimitError  = Class.new(ReadthisError)
+  UnknownSerializerError = Class.new(ReadthisError)
+end

data/lib/readthis/expanders.rb

@@ -7,7 +7,7 @@ module Readthis
       when key.is_a?(Array)
         key.flat_map { |elem| expand_key(elem) }.join('/')
       when key.is_a?(Hash)
-        key.sort_by { |key, _| key.to_s }.map { |key, val| "#{key}=#{val}" }.join('/')
+        key.sort_by { |hkey, _| hkey.to_s }.map { |hkey, val| "#{hkey}=#{val}" }.join('/')
       when key.respond_to?(:to_param)
         key.to_param
       else

data/lib/readthis/serializers.rb

@@ -0,0 +1,108 @@
+require 'json'
+require 'readthis/errors'
+require 'readthis/passthrough'
+
+module Readthis
+  class Serializers
+    BASE_SERIALIZERS = {
+      Marshal     => 0x1,
+      Passthrough => 0x2,
+      JSON        => 0x3
+    }.freeze
+
+    SERIALIZER_LIMIT = 7
+
+    attr_reader :serializers, :inverted
+
+    # Creates a new Readthis::Serializers entity. No configuration is expected
+    # during initialization.
+    #
+    def initialize
+      reset!
+    end
+
+    # Append a new serializer. Up to 7 total serializers may be configured for
+    # any single application be configured for any single application. This
+    # limit is based on the number of bytes available in the option flag.
+    #
+    # @param [Module] Any object that responds to `dump` and `load`
+    # @return [self] Returns itself for possible chaining
+    #
+    # @example
+    #
+    #     serializers = Readthis::Serializers.new
+    #     serializers << Oj
+    #
+    def <<(serializer)
+      case
+      when serializers.frozen?
+        fail SerializersFrozenError
+      when serializers.length > SERIALIZER_LIMIT
+        fail SerializersLimitError
+      else
+        @serializers[serializer] = flags.max.succ
+        @inverted = @serializers.invert
+      end
+
+      self
+    end
+
+    # Freeze the serializers hash, preventing modification.
+    #
+    def freeze!
+      serializers.freeze
+    end
+
+    # Reset the instance back to the default state. Useful for cleanup during
+    # testing.
+    #
+    def reset!
+      @serializers = BASE_SERIALIZERS.dup
+      @inverted = @serializers.invert
+    end
+
+    # Find a flag for a serializer object.
+    #
+    # @param [Object] Look up a flag by object
+    # @return [Number] Corresponding flag for the serializer object
+    # @raise [UnknownSerializerError] Indicates that a serializer was
+    #   specified, but hasn't been configured for usage.
+    #
+    # @example
+    #
+    #   serializers.assoc(JSON) #=> 1
+    #
+    def assoc(serializer)
+      flag = serializers[serializer]
+
+      unless flag
+        fail UnknownSerializerError, "'#{serializer}' hasn't been configured"
+      end
+
+      flag
+    end
+
+    # Find a serializer object by flag value.
+    #
+    # @param [Number] Flag to look up the serializer object by
+    # @return [Module] The serializer object
+    #
+    # @example
+    #
+    #   serializers.rassoc(1) #=> Marshal
+    #
+    def rassoc(flag)
+      inverted[flag & inverted.length]
+    end
+
+    # @private
+    def marshals
+      serializers.keys
+    end
+
+    # @private
+    def flags
+      serializers.values
+    end
+  end
+end

data/lib/readthis/version.rb

@@ -1,3 +1,3 @@
 module Readthis
-  VERSION = '0.8.1'
+  VERSION = '1.2.1'
 end

data/lib/readthis.rb

@@ -1,5 +1,42 @@
 require 'readthis/cache'
+require 'readthis/errors'
+require 'readthis/serializers'
 require 'readthis/version'
 
 module Readthis
+  extend self
+
+  # The current, global, instance of serializers that is used by all cache
+  # instances.
+  #
+  # @returns [Readthis::Serializers] An cached Serializers instance
+  #
+  # @see readthis/serializers
+  #
+  def serializers
+    @serializers ||= Readthis::Serializers.new
+  end
+
+  # Indicates whether connection error tolerance is enabled. With tolerance
+  # enabled every operation will return a `nil` value.
+  #
+  # @returns [Boolean] True for enabled, false for disabled
+  #
+  def fault_tolerant?
+    @fault_tolerant
+  end
+
+  # Toggle fault tolerance for connection errors.
+  #
+  # @param [Boolean] The new value for fault tolerance
+  #
+  def fault_tolerant=(value)
+    @fault_tolerant = value
+  end
+
+  # @private
+  def reset!
+    @fault_tolerant = nil
+    @serializers = nil
+  end
 end