readthis 0.3.0 → 0.4.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA1:
-  metadata.gz: dd2e3cdc15043b250cf043b45d05cc991297114b
-  data.tar.gz: 67ed2a3248bb47a61cef51f3d5af65c56a4d043b
+  metadata.gz: f5a03b37918d41dfd0e30151cb3049638df50aa8
+  data.tar.gz: 9c80690714866bc6992eafa9c137e785cf72a88d
 SHA512:
-  metadata.gz: 71eed4853e2cbf4d7e1832a8399ff06ed9ce3fbb2aca120f6ff10ec60c61470dedd96b1096d712a7df47fd47a828f91b60cd7d70c80de87bbdbb7d4d771f2264
-  data.tar.gz: abc9ce7f90b7fd0d4be2c340781bf1087b3f9d34a431dfcee81b5125d0c839d82b7cf703e06e3ed061ad8f2df43f63944f7297370d8fc2631f7d3a9d46dab888
+  metadata.gz: 9297ce7d8525ecdc2715399b3b657fb3cc12e581e4e9d067d762f7e9a134ffe6a35e48f6867bf7a26902f5dc48b6106e5c6dd012bc5e16372ad1b715fa6faa60
+  data.tar.gz: d431742cd9f908aabdd06dfeaab2ed99b7165158fff51c57c947d9857d336e2a5c82fe42edae15a41c7cdd8043b3fc950c09e6fd60feb4d69c571fb71e2fe6e3

data/CHANGELOG.md

@@ -1,3 +1,11 @@
+## v0.4.0 2014-12-11
+
+- Added: Force the use of `hiredis` as the adapter. It is dramatically faster,
+  but prevents the project from being used in `jruby`. If we get interest from
+  some `jruby` projects we can soften the requirement.
+- Added: Compression! Adheres to the `ActiveSupport::Store` documentation.
+- Fixed: Gracefully handle `nil` passed as `options` to any cache method.
+
 ## v0.3.0 2014-12-01
 
 - Added: Use `to_param` for key expansion, only when available. Makes it

data/PERFORMANCE.md

@@ -0,0 +1,73 @@
+Results from the various benchmarks in `./bencharks`. Hardware doesnt't matter
+much, as we're simply looking for a comparison against other libraries and prior
+verions.
+
+## Footprint
+
+Footprint compared to `redis-activesupport`:
+
+```
+# Total allocated objects after require
+readthis: 20602
+redis-activesupport: 78630
+```
+
+## Performance
+
+Performance compared to `dalli` and `redis-activesupport`:
+
+```
+Raw Read Multi:
+Calculating -------------------------------------
+ readthis:read-multi   500.000  i/100ms
+  redisas:read-multi    95.000  i/100ms
+    dalli:read-multi    97.000  i/100ms
+-------------------------------------------------
+ readthis:read-multi      5.286k (± 2.7%) i/s -     26.500k
+  redisas:read-multi    959.405  (± 4.2%) i/s -      4.845k
+    dalli:read-multi    978.803  (± 2.1%) i/s -      4.947k
+
+Comparison:
+ readthis:read-multi:     5286.0 i/s
+    dalli:read-multi:      978.8 i/s - 5.40x slower
+  redisas:read-multi:      959.4 i/s - 5.51x slower
+
+Raw Fetch Multi:
+Calculating -------------------------------------
+readthis:fetch-multi   448.000  i/100ms
+ redisas:fetch-multi    84.000  i/100ms
+   dalli:fetch-multi    99.000  i/100ms
+-------------------------------------------------
+readthis:fetch-multi      4.682k (± 2.4%) i/s -     23.744k
+ redisas:fetch-multi    848.101  (± 3.2%) i/s -      4.284k
+   dalli:fetch-multi      1.006k (± 2.4%) i/s -      5.049k
+
+Comparison:
+readthis:fetch-multi:     4682.4 i/s
+   dalli:fetch-multi:     1005.6 i/s - 4.66x slower
+ redisas:fetch-multi:      848.1 i/s - 5.52x slower
+
+Compressed Writes:
+Calculating -------------------------------------
+      readthis:write     1.003k i/100ms
+         dalli:write   913.000  i/100ms
+-------------------------------------------------
+      readthis:write     11.095k (± 5.7%) i/s -     56.168k
+         dalli:write      9.829k (± 1.8%) i/s -     49.302k
+
+Comparison:
+      readthis:write:    11095.5 i/s
+         dalli:write:     9828.8 i/s - 1.13x slower
+
+Compressed Read Multi:
+Calculating -------------------------------------
+ readthis:read_multi   446.000  i/100ms
+    dalli:read_multi    97.000  i/100ms
+-------------------------------------------------
+ readthis:read_multi      4.728k (± 4.6%) i/s -     23.638k
+    dalli:read_multi    985.986  (± 3.9%) i/s -      4.947k
+
+Comparison:
+ readthis:read_multi:     4728.3 i/s
+    dalli:read_multi:      986.0 i/s - 4.80x slower
+```

data/README.md

@@ -13,46 +13,7 @@ behavior for `fetch_multi`.
 
 ## Footprint & Performance
 
-Footprint compared to `redis-activesupport`:
-
-```
-# Total allocated objects after require
-readthis: 19,964
-redis-activesupport: 78,630
-```
-
-Performance compared to `dalli` and `redis-activesupport` for \*multi
-operations:
-
-```
-Calculating -------------------------------------
- readthis:read-multi   118.000  i/100ms
-  redisas:read-multi    94.000  i/100ms
-    dalli:read-multi    92.000  i/100ms
--------------------------------------------------
- readthis:read-multi      1.206k (± 4.6%) i/s -      6.018k
-  redisas:read-multi    973.086  (± 4.4%) i/s -      4.888k
-    dalli:read-multi    949.348  (± 4.1%) i/s -      4.784k
-
-Comparison:
- readthis:read-multi:     1206.0 i/s
-  redisas:read-multi:      973.1 i/s - 1.24x slower
-    dalli:read-multi:      949.3 i/s - 1.27x slower
-
-Calculating -------------------------------------
-readthis:fetch-multi   114.000  i/100ms
- redisas:fetch-multi    82.000  i/100ms
-   dalli:fetch-multi    97.000  i/100ms
--------------------------------------------------
-readthis:fetch-multi      1.157k (± 5.0%) i/s -      5.814k
- redisas:fetch-multi    829.211  (± 4.2%) i/s -      4.182k
-   dalli:fetch-multi    979.081  (± 3.8%) i/s -      4.947k
-
-Comparison:
-readthis:fetch-multi:     1157.2 i/s
-   dalli:fetch-multi:      979.1 i/s - 1.18x slower
- redisas:fetch-multi:      829.2 i/s - 1.40x slower
-```
+See [Performance][PERFORMANCE.md]
 
 ## Installation
 
@@ -86,10 +47,24 @@ You'll want to use a specific database for caching, just in case you need to
 clear the cache entirely. Appending a number between 0 and 15 will specify the
 redis database, which defaults to 0. For example, using database 2:
 
-```
+```bash
 REDIS_URL=redis://localhost:6379/2
 ```
 
+Compression can be enabled for all actions by passing the `compress` flag. By
+default all values greater than 1024k will be compressed automatically. If there
+is any content has not been stored with compression, or perhaps was compressed
+but is beneath the compression threshold, it will be passed through as is. This
+means it is safe to enable or change compression with an existing cache. There
+will be a decoding performance penalty in this case, but it should be minor.
+
+```ruby
+config.cache_store = :readthis_store, ENV.fetch('REDIS_URL'), {
+  compress: true,
+  compression_threshold: 2.kilobytes
+}
+```
+
 ## Differences
 
 Readthis supports all of standard cache methods except for the following:

data/benchmarks/compressed.rb

@@ -0,0 +1,73 @@
+require 'bundler'
+
+Bundler.setup
+
+require 'benchmark/ips'
+require 'dalli'
+require 'active_support'
+require 'active_support/cache/dalli_store'
+require 'readthis'
+
+dalli = ActiveSupport::Cache::DalliStore.new(
+  'localhost',
+  pool_size: 5,
+  compressed: true,
+  compression_threshold: 8
+)
+
+readthis  = Readthis::Cache.new(
+  'redis://localhost:6379/11',
+  pool_size: 5,
+  compressed: true,
+  compression_threshold: 128
+)
+
+KEY  = 'key'
+TEXT = <<-TEXT
+  An abstract cache store class. There are multiple cache store implementations, each having its own additional features. See the classes under the ActiveSupport::Cache module, e.g. ActiveSupport::Cache::MemCacheStore. MemCacheStore is currently the most popular cache store for large production websites.
+  Some implementations may not support all methods beyond the basic cache methods of fetch, write, read, exist?, and delete.
+  ActiveSupport::Cache::Store can store any serializable Ruby object.
+  cache = ActiveSupport::Cache::MemoryStore.new
+  cache.read('city')   # => nil
+  cache.write('city', "Duckburgh")
+  cache.read('city')   # => "Duckburgh"
+  Keys are always translated into Strings and are case sensitive. When an object is specified as a key and has a cache_key method defined, this method will be called to define the key. Otherwise, the to_param method will be called. Hashes and Arrays can also be used as keys. The elements will be delimited by slashes, and the elements within a Hash will be sorted by key so they are consistent.
+  cache.read('city') == cache.read(:city)   # => true
+  Nil values can be cached.
+  If your cache is on a shared infrastructure, you can define a namespace for your cache entries. If a namespace is defined, it will be prefixed on to every key. The namespace can be either a static value or a Proc. If it is a Proc, it will be invoked when each key is evaluated so that you can use application logic to invalidate keys.
+  cache.namespace = -> { @last_mod_time }  # Set the namespace to a variable
+  @last_mod_time = Time.now  # Invalidate the entire cache by changing namespace
+  Caches can also store values in a compressed format to save space and reduce time spent sending data. Since there is overhead, values must be large enough to warrant compression. To turn on compression either pass compress: true in the initializer or as an option to fetch or write. To specify the threshold at which to compress values, set the :compress_threshold option. The default threshold is 16K.
+TEXT
+
+puts 'Compressed Writes:'
+Benchmark.ips do |x|
+  x.report 'readthis:write' do
+    readthis.write(KEY, TEXT)
+  end
+
+  x.report 'dalli:write' do
+    dalli.write(KEY, TEXT)
+  end
+
+  x.compare!
+end
+
+puts 'Compressed Read Multi:'
+MULTI_KEY = (1..30).to_a
+MULTI_KEY.each do |key|
+  readthis.write(key, TEXT)
+  dalli.write(key, TEXT)
+end
+
+Benchmark.ips do |x|
+  x.report 'readthis:read_multi' do
+    readthis.read_multi(*MULTI_KEY)
+  end
+
+  x.report 'dalli:read_multi' do
+    dalli.read_multi(*MULTI_KEY)
+  end
+
+  x.compare!
+end

data/benchmarks/multi.rb

@@ -8,15 +8,17 @@ require 'redis-activesupport'
 require 'active_support/cache/dalli_store'
 require 'readthis'
 
-redis_url = 'redis://localhost:6379/11'
-dalli     = ActiveSupport::Cache::DalliStore.new('localhost', namespace: 'da', pool_size: 5)
-redisas   = ActiveSupport::Cache::RedisStore.new(redis_url + '/ra')
-readthis  = Readthis::Cache.new(redis_url, namespace: 'rd', expires_in: 60)
+REDIS_URL = 'redis://localhost:6379/11'
+dalli     = ActiveSupport::Cache::DalliStore.new('localhost', namespace: 'da', pool_size: 5, expires_in: 60)
+redisas   = ActiveSupport::Cache::RedisStore.new(REDIS_URL + '/ra', expires_in: 60)
+readthis  = Readthis::Cache.new(REDIS_URL, namespace: 'rd', expires_in: 60)
 
 ('a'..'z').each do |key|
-  dalli.write(key, key * 1024)
-  readthis.write(key, key * 1024)
-  redisas.write(key, key * 1024)
+  value = key * 1024
+
+  dalli.write(key, value)
+  readthis.write(key, value)
+  redisas.write(key, value)
 end
 
 Benchmark.ips do |x|

data/lib/readthis/cache.rb

@@ -1,11 +1,19 @@
+require 'readthis/compressor'
 require 'readthis/expanders'
 require 'readthis/notifications'
 require 'redis'
+require 'hiredis'
 require 'connection_pool'
 
 module Readthis
   class Cache
-    attr_reader :expires_in, :namespace, :pool
+    attr_reader :compress,
+      :compression_threshold,
+      :expires_in,
+      :namespace,
+      :pool
+
+    alias_method :compress?, :compress
 
     # Provide a class level lookup of the proper notifications module.
     # Instrumention is expected to occur within applications that have
@@ -22,19 +30,34 @@ module Readthis
     # Creates a new Readthis::Cache object with the given redis URL. The URL
     # is parsed by the redis client directly.
     #
+    # @param url [String] A redis compliant url with necessary connection details
+    # @option options [String] :namespace Prefix used to namespace entries
+    # @option options [Number] :expires_in The number of seconds until an entry expires
+    # @option options [Boolean] :compress Enable or disable automatic compression
+    # @option options [Number] :compression_threshold The size a string must be for compression
+    #
+    # @example Create a new cache instance
     #   Readthis::Cache.new('redis://localhost:6379/0', namespace: 'cache')
+    #
+    # @example Create a compressed cache instance
+    #   Readthis::Cache.new('redis://localhost:6379/0', compress: true, compression_threshold: 2048)
+    #
     def initialize(url, options = {})
       @expires_in = options.fetch(:expires_in, nil)
-      @namespace  = options.fetch(:namespace, nil)
+      @namespace  = options.fetch(:namespace,  nil)
+      @compress   = options.fetch(:compress,   false)
+      @compression_threshold = options.fetch(:compression_threshold, 1024)
 
       @pool = ConnectionPool.new(pool_options(options)) do
-        Redis.new(url: url)
+        Redis.new(url: url, driver: :hiredis)
       end
     end
 
     def read(key, options = {})
       invoke(:read, key) do |store|
-        store.get(namespaced_key(key, merged_options(options)))
+        value = store.get(namespaced_key(key, merged_options(options)))
+
+        decompressed(value)
       end
     end
 
@@ -44,9 +67,9 @@ module Readthis
 
       invoke(:write, key) do |store|
         if expiration = options[:expires_in]
-          store.setex(namespaced, expiration, value)
+          store.setex(namespaced, expiration, compressed(value))
         else
-          store.set(namespaced, value)
+          store.set(namespaced, compressed(value))
         end
       end
     end
@@ -85,7 +108,9 @@ module Readthis
       mapping = keys.map { |key| namespaced_key(key, options) }
 
       invoke(:read_multi, keys) do |store|
-        keys.zip(store.mget(mapping)).to_h
+        values = decompressed_multi(store.mget(mapping))
+
+        keys.zip(values).to_h
       end
     end
 
@@ -127,6 +152,34 @@ module Readthis
 
     private
 
+    def compressed(value)
+      if compress?
+        compressor.compress(value)
+      else
+        value
+      end
+    end
+
+    def decompressed(value)
+      if compress?
+        compressor.decompress(value)
+      else
+        value
+      end
+    end
+
+    def decompressed_multi(values)
+      if compress?
+        values.map { |value| compressor.decompress(value) }
+      else
+        values
+      end
+    end
+
+    def compressor
+      @compressor ||= Readthis::Compressor.new(threshold: compression_threshold)
+    end
+
     def instrument(operation, key)
       name    = "cache_#{operation}.active_support"
       payload = { key: key }
@@ -145,6 +198,7 @@ module Readthis
     end
 
     def merged_options(options)
+      options = options || {}
       options[:namespace]  ||= namespace
       options[:expires_in] ||= expires_in
       options
@@ -156,7 +210,7 @@ module Readthis
     end
 
     def namespaced_key(key, options)
-      Readthis::Expanders.expand(key, options[:namespace])
+      Readthis::Expanders.namespace_key(key, options[:namespace])
     end
   end
 end

data/lib/readthis/compressor.rb

@@ -0,0 +1,41 @@
+require 'zlib'
+
+module Readthis
+  class Compressor
+    attr_reader :threshold
+
+    # Create a new Readthis::Compressor object that pivots on the provided
+    # threshold value.
+    #
+    # @param threshold [Number] the threshold size required for compression
+    def initialize(threshold: 1024)
+      @threshold = threshold
+    end
+
+    # Compress a value if its size is greater or equal to the current threshold.
+    #
+    # @param value [String] a string to compress
+    def compress(value)
+      if value.size >= threshold
+        Zlib::Deflate.deflate(value)
+      else
+        value
+      end
+    end
+
+    # Decompress a previously compressed object. It will attempt to decode a
+    # value regardless of whether it has been compressed, but will rescue
+    # decoding errors.
+    #
+    # @param value [String] a possibly compressed string to decompress
+    def decompress(value)
+      if value.size >= threshold
+        Zlib::Inflate.inflate(value)
+      else
+        value
+      end
+    rescue Zlib::Error
+      value
+    end
+  end
+end

data/lib/readthis/expanders.rb

@@ -1,20 +1,28 @@
 module Readthis
   module Expanders
-    def self.expand(key, namespace = nil)
-      expanded =
-        if key.respond_to?(:cache_key)
-          key.cache_key
-        elsif key.is_a?(Array)
-          key.flat_map { |elem| expand(elem) }.join('/')
-        elsif key.is_a?(Hash)
-          key.sort_by { |key, _| key.to_s }.map { |key, val| "#{key}=#{val}" }.join('/')
-        elsif key.respond_to?(:to_param)
-          key.to_param
-        else
-          key
-        end
+    def self.expand_key(key)
+      case
+      when key.respond_to?(:cache_key)
+        key.cache_key
+      when key.is_a?(Array)
+        key.flat_map { |elem| expand_key(elem) }.join('/')
+      when key.is_a?(Hash)
+        key.sort_by { |key, _| key.to_s }.map { |key, val| "#{key}=#{val}" }.join('/')
+      when key.respond_to?(:to_param)
+        key.to_param
+      else
+        key
+      end
+    end
+
+    def self.namespace_key(key, namespace)
+      expanded = expand_key(key)
 
-      namespace ? "#{namespace}:#{expanded}" : expanded
+      if namespace
+        "#{namespace}:#{expanded}"
+      else
+        expanded
+      end
     end
   end
 end

data/lib/readthis/version.rb

@@ -1,3 +1,3 @@
 module Readthis
-  VERSION = '0.3.0'
+  VERSION = '0.4.0'
 end

data/readthis.gemspec

@@ -18,6 +18,7 @@ Gem::Specification.new do |spec|
   spec.require_paths = ['lib']
 
   spec.add_dependency 'redis',           '~> 3.0'
+  spec.add_dependency 'hiredis',         '~> 0.5'
   spec.add_dependency 'connection_pool', '~> 2.1'
 
   spec.add_development_dependency 'bundler'

data/spec/readthis/cache_spec.rb

@@ -20,6 +20,17 @@ RSpec.describe Readthis::Cache do
 
       expect(cache.expires_in).to eq(10)
     end
+
+    it 'stores compression parameters' do
+      cache = Readthis::Cache.new(
+        url,
+        compress: true,
+        compression_threshold: 8
+      )
+
+      expect(cache.compress).to be_truthy
+      expect(cache.compression_threshold).to eq(8)
+    end
   end
 
   describe '#write' do
@@ -53,6 +64,43 @@ RSpec.describe Readthis::Cache do
     end
   end
 
+  describe '#read' do
+    it 'gracefully handles nil options' do
+      expect { cache.read('whatever', nil) }.not_to raise_error
+    end
+  end
+
+  describe '#compress' do
+    it 'round trips entries when compression is enabled' do
+      com_cache = Readthis::Cache.new(url, compress: true, compression_threshold: 8)
+      raw_cache = Readthis::Cache.new(url)
+      value = 'enough text that it should be compressed'
+
+      com_cache.write('compressed', value)
+
+      expect(raw_cache.read('compressed')).not_to eq(value)
+      expect(com_cache.read('compressed')).to eq(value)
+    end
+
+    it 'round trips bulk entries when compression is enabled' do
+      cache = Readthis::Cache.new(url, compress: true, compression_threshold: 8)
+      value = 'also enough text to compress'
+
+      cache.write('comp-a', value)
+      cache.write('comp-b', value)
+
+      expect(cache.read_multi('comp-a', 'comp-b')).to eq(
+        'comp-a' => value,
+        'comp-b' => value
+      )
+
+      expect(cache.fetch_multi('comp-a', 'comp-b')).to eq(
+        'comp-a' => value,
+        'comp-b' => value
+      )
+    end
+  end
+
   describe '#fetch' do
     it 'gets an existing value' do
       cache.write('great-key', 'great')

data/spec/readthis/compressor_spec.rb

@@ -0,0 +1,38 @@
+require 'readthis/compressor'
+
+RSpec.describe Readthis::Compressor do
+  describe '#compress' do
+    it 'compresses the input' do
+      compressor   = Readthis::Compressor.new(threshold: 0)
+      input      = 'aaa bbb ccc'
+      output     = compressor.compress(input)
+
+      expect(input).not_to eq(output)
+    end
+
+    it 'passes input below the threshold size through uncompressed' do
+      compressor = Readthis::Compressor.new(threshold: 1024)
+      input      = 'abcdefg'
+
+      expect(compressor.compress(input)).to eq(input)
+    end
+  end
+
+  describe '#decompress' do
+    it 'decompresses compressed data' do
+      compressor   = Readthis::Compressor.new(threshold: 0)
+      input        = 'aaa bbb ccc'
+      compressed   = compressor.compress(input)
+      decompressed = compressor.decompress(compressed)
+
+      expect(decompressed).to eq(input)
+    end
+
+    it 'passes through decompression failures' do
+      compressor = Readthis::Compressor.new(threshold: 0)
+      input      = 'abcdefg'
+
+      expect(compressor.decompress(input)).to eq(input)
+    end
+  end
+end

data/spec/readthis/expanders_spec.rb

@@ -2,7 +2,7 @@ require 'readthis/expanders'
 
 RSpec.describe Readthis::Expanders do
   def expand(key, namespace = nil)
-    Readthis::Expanders.expand(key, namespace)
+    Readthis::Expanders.namespace_key(key, namespace)
   end
 
   describe '#expand' do

metadata

@@ -1,14 +1,14 @@
 --- !ruby/object:Gem::Specification
 name: readthis
 version: !ruby/object:Gem::Version
-  version: 0.3.0
+  version: 0.4.0
 platform: ruby
 authors:
 - Parker Selbert
 autorequire: 
 bindir: bin
 cert_chain: []
-date: 2014-12-02 00:00:00.000000000 Z
+date: 2014-12-11 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: redis
@@ -24,6 +24,20 @@ dependencies:
     - - "~>"
       - !ruby/object:Gem::Version
         version: '3.0'
+- !ruby/object:Gem::Dependency
+  name: hiredis
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '0.5'
+  type: :runtime
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '0.5'
 - !ruby/object:Gem::Dependency
   name: connection_pool
   requirement: !ruby/object:Gem::Requirement
@@ -94,19 +108,23 @@ files:
 - CONTRIBUTING.md
 - Gemfile
 - LICENSE.txt
+- PERFORMANCE.md
 - README.md
 - Rakefile
+- benchmarks/compressed.rb
 - benchmarks/memory.rb
 - benchmarks/multi.rb
 - bin/rspec
 - lib/active_support/cache/readthis_store.rb
 - lib/readthis.rb
 - lib/readthis/cache.rb
+- lib/readthis/compressor.rb
 - lib/readthis/expanders.rb
 - lib/readthis/notifications.rb
 - lib/readthis/version.rb
 - readthis.gemspec
 - spec/readthis/cache_spec.rb
+- spec/readthis/compressor_spec.rb
 - spec/readthis/expanders_spec.rb
 - spec/readthis/notifications_spec.rb
 - spec/spec_helper.rb
@@ -136,6 +154,7 @@ specification_version: 4
 summary: Pooled active support compliant caching with redis
 test_files:
 - spec/readthis/cache_spec.rb
+- spec/readthis/compressor_spec.rb
 - spec/readthis/expanders_spec.rb
 - spec/readthis/notifications_spec.rb
 - spec/spec_helper.rb