redis-native_hash 0.1.0 → 0.2.1

data/README.mkd

@@ -1,8 +1,146 @@
-= redis-native_hash
+# redis-native_hash
 
-Description goes here.
+Tools to help expose Redis' powerful Hash type through a familiar Ruby `Hash` interface.
+`NativeHash` provides a general solution for exposing reasonably sized Redis hashes as
+Ruby hashes, including sane and transparent transactions, nested hash support, and automatic
+serialization of complex data types. `BigHash` is provided to
+efficiently handle big or even enormous Redis hashes. `LazyHash` is a convenient proxy
+for `NativeHash` useful when you aren't sure the hash will be read (useful for sessions).
 
-== Contributing to redis-native_hash
+Also included is Rack middleware to store sessions in Redis hashes, and a two Rails caching
+solution, one using Redis hashes and the other using Redis strings.
+
+## Example usage for `NativeHash`
+
+```ruby
+require "redis_hash"
+
+# Create a Ruby hash backed by Redis
+hash = Redis::NatveHash.new  # => {}
+hash[:foo] = :bar
+hash.key                     # => "20120512181125.368617.04d2abae82db62ece82b3805b654082b"
+hash.save                    # => true
+
+# Retrieve an existing hash from Redis
+existing = Redis::NativeHash.find(hash.key) # => {"foo" => :bar}
+
+# Symbols and strings can be used interchangeably, sort of like HashWithIndifferentAccess
+existing[:foo]    # => :bar
+existing["foo"]   # => :bar
+
+# Convert existing hash to Redis backed hash
+hash = {yin: "yang"}
+redis_hash = Redis::NativeHash.new.update(hash)
+
+# Create a hash with a custom key
+hash.key = :custom_key
+hash.key  # => :custom_key
+
+# Use namespaces
+hash = Redis::NativeHash.new(:custom_namespace)
+hash.namespace  # => :custom_namespace
+hash.key        # => "20120512212206.376929.5194d9ea37e2d1b6c773b860cce58c7d"
+hash.redis_key  # => "custom_namespace:20120512212206.376929.5194d9ea37e2d1b6c773b860cce58c7d"
+
+# Initialize with custom namespace and key
+hash = Redis::NativeHash.new(custom_namespace: "my_hash_key")
+hash[:test] = "value"
+hash.namespace  # => :custom_namespace
+hash.key        # => "my_hash_key"
+hash.redis_key  # => "custom_namespace:my_hash_key"
+hash.save       # => true
+
+# Retrieve existing hash using namespace and key
+existing = Redis::NativeHash.find(custom_namespace: "my_hash_key")  # => {"test" => "value"}
+```
+
+## Example usage for `BigHash`
+
+```ruby
+# Initializing a BigHash
+big = Redis::BigHash.new  # => #<Redis::BigHash:0x007fcdfc8890d8 @key=nil, @namespace=nil>
+big = Redis::BigHash.new("custom_key")
+big = Redis::BigHash.new("custom_key", "app_namespace")
+
+# No #save method as writes take place immediately
+big = Redis::BigHash("my_key")
+big.[:test] = "right now"
+redis.hget("my_key", "test")  # => "right_now"
+```
+
+## Usage for `LazyHash`
+
+A simple lazy-loading proxy object that should behave identically to NativeHash. 
+Check `hash.loaded?` if you need to know whether the underlying hash has been read.
+
+## Using as a Rails session store
+
+Change your `config/initializers/session_store.rb` to look something like this:
+```ruby
+require "redis_hash"
+MyAppName::Application.config.session_store :redis_hash
+```
+
+## Using Redis string-based caching implementation
+
+Only the string based implementation is able to properly handle automatic expiration, so it is preferred.
+
+Add the following line to the appropriate environment config in `config/environments/`
+
+```ruby
+config.cache_store = :redis_store
+```
+
+To set a cache expiration, use a line like this:
+
+```ruby
+config.cache_store = [:redis_store, :expires_in => 24.hours]
+```
+
+## Client helpers
+
+This gem adds a useful `Redis::ClientHelper` module to simplify both connection sharing and using multiple connections.
+Using `Redis::Client.default=` you can set the connection all future instances of `NativeHash`/`BigHash`/`LazyHash`
+will use.
+
+```ruby
+# Changes to the default cascade down, unless class-level defaults have already been set
+Redis::Client.default     # => #<Redis client v2.2.2 connected to redis://127.0.0.1:6379/0 (Redis v2.4.6)>
+redis = Redis.new(db: 8)  # => #<Redis client v2.2.2 connected to redis://127.0.0.1:6379/8 (Redis v2.4.6)>
+Redis::Client.default = redis
+Redis::Client.default     # => #<Redis client v2.2.2 connected to redis://127.0.0.1:6379/8 (Redis v2.4.6)>
+Redis::BigHash.redis      # => #<Redis client v2.2.2 connected to redis://127.0.0.1:6379/8 (Redis v2.4.6)>
+Redis::BigHash.new.redis  # => #<Redis client v2.2.2 connected to redis://127.0.0.1:6379/8 (Redis v2.4.6)>
+```
+
+The client helper also lets you set the redis connection to use for an entire class, or a single instance.
+
+```ruby
+Redis::BigHash.redis = Redis.new(db: 4)
+Redis::BigHash.redis      # => #<Redis client v2.2.2 connected to redis://127.0.0.1:6379/4 (Redis v2.4.6)>
+Redis::Client.default     # => #<Redis client v2.2.2 connected to redis://127.0.0.1:6379/0 (Redis v2.4.6)>
+hash = Redis::BigHash.new
+hash.redis = Redis.new(db: 5)
+hash.redis                # => #<Redis client v2.2.2 connected to redis://127.0.0.1:6379/5 (Redis v2.4.6)>
+Redis::BigHash.redis      # => #<Redis client v2.2.2 connected to redis://127.0.0.1:6379/4 (Redis v2.4.6)>
+```
+
+You can include the client helper into your own classes to give your own classes similar behavior.
+```ruby
+class CustomClass
+  include Redis::ClientHelper
+end
+CustomClass.redis         # => #<Redis client v2.2.2 connected to redis://127.0.0.1:6379/0 (Redis v2.4.6)>
+x = CustomClass.new
+x.redis = Redis.new(db: 3)
+x.redis                   # => #<Redis client v2.2.2 connected to redis://127.0.0.1:6379/3 (Redis v2.4.6)>
+```
+
+**Note:** Your own classes will use `Redis::Client.default` unless a class-level or instance-level connection is set.
+
+**See tests for more examples.**
+
+## Contributing to redis-native_hash
 
 * Check out the latest master to make sure the feature hasn't been implemented or the bug hasn't been fixed yet
 * Check out the issue tracker to make sure someone already hasn't requested it and/or contributed it
@@ -10,10 +148,7 @@ Description goes here.
 * Start a feature/bugfix branch
 * Commit and push until you are happy with your contribution
 * Make sure to add tests for it. This is important so I don't break it in a future version unintentionally.
-* Please try not to mess with the Rakefile, version, or history. If you want to have your own version, or is otherwise necessary, that is fine, but please isolate to its own commit so I can cherry-pick around it.
-
-== Copyright
 
-Copyright (c) 2011 Lyconic. See LICENSE.txt for
-further details.
+### Copyright
 
+Copyright (c) 2011 Lyconic. See LICENSE.txt for further details.

data/TODO.mkd

@@ -0,0 +1,58 @@
+## Plan to make a replacement for Rack::Session::Abstract::SessionHash ##
+
+### Rationale ###
+
+`SessionHash` attempts to lazy load the session in Rack 1.3 and higher.
+It does so in a way that does not play nice with Redis::NativeHash.
+Specifically, it `#merge!`s the content of the hash returned by the redis
+session store, instead of using that hash as the actual session hash.
+Since the session hash is no longer a `Redis::NativeHash` instance it is unable
+to keep track of what session values have been changed. This results in an
+additional read when the session is saved since NativeHash needs to read from
+redis again to figure out what values have changed.
+
+### Alternatives ###
+
+1. `NativeHash` could be changed to allow for blind writes. There are two major
+downsides to this approach.
+
+  1. `NativeHash` would have no way of knowing which keys are already in the redis
+  hash without reading from redis a second time. Not reading the hash before
+  writing would mean you would have to delete the hash before beginnning to
+  write the data to ensure any keys which no longer exist have been removed.
+
+  2. This would circumvent NativeHash's built-in support for concurrent writes
+  and could lead to strange issues with multiple apps have to write to the
+  same session.
+
+2. Leave it be. Nothing is technically "broken", so don't fix it.
+
+  1. Session read/write cycles will require 2x the number of complete loads
+  of the corresponding redis hash.
+
+  2. Creating and destroying `NativeHash` instances could result in needless
+  memory usage and garbage collection on every page request.
+  The total cost of this is probably pretty small.
+
+3. Just overwrite `Rack::Session::Abstract::ID` and make `env['rack.session']`
+an instance of `Redis::NativeHash`. You'll lose lazy loading, but it will be a
+quick fix to the extra read problem. This probably won't work though because
+other methods on `Abstract::ID` look for methods like `#loaded?`... ok, it will
+work, it will just require several methods of `Abstract::ID` to be overwritten.
+May actually be easier to just replace `Abstract::ID` altogether.
+
+
+### Proposal ###
+
+Take the time to write a drop in replacement for `SessionHash`.
+This replacement would respond to the special methods added to `SessionHash`
+and will also accomplish similar lazy loading for the underlying `NativeHash`.
+
+Attempt to integrate with existing `Rack::Session::Abstract::ID` by overriding
+`ID#prepare_session`.
+
+## Make RedisHashSession support expirations ##
+
+That's right, it currently doesn't support expirations.
+Make it happen and test it. Provide a default expiration too, probably.
+

data/VERSION

@@ -1 +1 @@
-0.1.0
+0.2.1

data/lib/action_dispatch/session/redis_hash.rb

@@ -0,0 +1,4 @@
+class ActionDispatch::Session::RedisHash < ActionDispatch::Session::AbstractStore
+  include ::Redis::RedisHashSession
+end
+

data/lib/active_support/cache/redis_hash.rb

@@ -0,0 +1,48 @@
+require 'redis/native_hash'
+
+module ActiveSupport
+  module Cache
+    class RedisHash < Store
+      def initialize(*options)
+        options = options.extract_options!
+        super(options)
+        @hash = ::Redis::BigHash.new(options[:key], options[:namespace] || :rails_cache)
+        extend Strategy::LocalCache
+      end
+
+      # Reads multiple values from the cache using a single call to the
+      # servers for all keys.
+      def read_multi(*names)
+        @hash[*names]
+      end
+
+      # Clear the entire cache on server. This method should
+      # be used with care when shared cache is being used.
+      def clear(options = nil)
+        @hash.destroy
+      end
+
+      protected
+
+        # Read an entry from the cache.
+        def read_entry(key, options)
+          @hash[key]
+        end
+
+        # Write an entry to the cache.
+        def write_entry(key, entry, options)
+          if options && options[:unless_exist]
+            @hash.add(key, entry)
+          else
+            @hash[key] = entry
+          end
+        end
+
+        # Delete an entry from the cache.
+        def delete_entry(key, options)
+          @hash.delete(key)
+        end
+    end
+  end
+end
+

data/lib/active_support/cache/redis_store.rb

@@ -0,0 +1,46 @@
+module ActiveSupport
+  module Cache
+    class RedisStore < Store
+      include Redis::ClientHelper
+      def initialize(*options)
+        options = options.extract_options!
+        super(options)
+        extend Strategy::LocalCache
+      end
+
+      # Reads multiple values from the cache using a single call to the
+      # servers for all keys.
+      def read_multi(*names)
+        values = redis.mget *names
+        values.map{ |v| Redis::Marshal.load(v) }
+      end
+
+      # Clear the entire cache on server. This method should
+      # be used with care when shared cache is being used.
+      def clear(options = nil)
+        redis.flushdb
+      end
+
+      protected
+
+        # Read an entry from the cache.
+        def read_entry(key, options)
+          Redis::Marshal.load(redis.get(key))
+        end
+
+        # Write an entry to the cache.
+        def write_entry(key, entry, options)
+          method = options && options[:unless_exist] ? :setnx : :set
+          expires_in = options[:expires_in].to_i
+          redis.send(method, key, Redis::Marshal.dump(entry))
+          redis.expire(key, expires_in) if expires_in > 0
+        end
+
+        # Delete an entry from the cache.
+        def delete_entry(key, options)
+          redis.del(key)
+        end
+    end
+  end
+end
+

data/lib/rack/session/redis_hash.rb

@@ -1,34 +1,69 @@
-module Rack
-  module Session
-    class RedisHash < Abstract::ID
+module Redis::RedisHashSession
 
-      def get_session(env, sid)
-        session = Redis::NativeHash.find session_prefix => sid
-        unless sid and session
-          env['rack.errors'].puts("Session '#{sid.inspect}' not found, initializing...") if $VERBOSE and not sid.nil?
-          session = Redis::NativeHash.new session_prefix
-          sid = session.key
-        end
-        return [sid, session]
-      end
+  def initialize(app, options = {})
+    super
+    @expire_after = options[:expire_after] || 60*60 # 60 minutes, default
+  end
 
-      def set_session(env, session_id, session, options)
-        if options[:drop]
-          session.destroy
-          return false if options[:drop]
-        end
-        if options[:renew]
-          session_id = session.renew_key
-        end
-        session.save
-        return session_id
-      end
+  def get_session(env, sid)
+    session = Redis::LazyHash.new session_prefix => sid
+    sid = session.key
+    return [sid, session]
+  end
 
-      def session_prefix
-        :rack_session
-      end
+  def set_session(env, session_id, session, options)
+    @expire_after = options[:expire_after] || @expire_after
+    unless session.kind_of?(Redis::LazyHash)
+      real_session = Redis::LazyHash.new(session_prefix)
+      real_session.update(session) if session.kind_of?(Hash)
+      real_session.key = session_id unless session_id.nil?
+      session = real_session
+    end
+    if options[:drop]
+      session.destroy
+      return false if options[:drop]
+    end
+    if options[:renew]
+      session_id = session.renew_key
+    end
+    session.save
+    session.expire @expire_after
+    return session_id
+  end
 
+  def destroy_session(env, sid, options)
+    session = Redis::LazyHash.new( session_prefix => sid )
+    unless session.nil?
+      options[:renew] ? session.renew_key : session.destroy
+      session.key
     end
   end
+
+  def session_prefix
+    :rack_session
+  end
 end
 
+module Rack
+  module Session
+    module Abstract
+      class ID
+        # overwrite prepare_session behavior to turn off use of SessionHash
+        def prepare_session(env)
+          session_was                  = env[ENV_SESSION_KEY]
+          env[ENV_SESSION_OPTIONS_KEY] = OptionsHash.new(self, env, @default_options)
+          env[ENV_SESSION_OPTIONS_KEY][:id], env[ENV_SESSION_KEY] = load_session(env)
+          env[ENV_SESSION_KEY].merge! session_was if session_was
+        end
+      end
+    end
+  end
+end
+
+module Rack
+  module Session
+    class RedisHash < Abstract::ID
+      include ::Redis::RedisHashSession
+    end
+  end
+end

data/lib/redis/big_hash.rb

@@ -0,0 +1,105 @@
+require 'securerandom'
+
+class Redis
+  class BigHash
+    include ClientHelper
+    include KeyHelper
+
+    attr_reader :namespace
+
+    def initialize( key = nil, namespace = nil )
+      @key       = key
+      @namespace = namespace
+      super(nil)
+    end
+
+    def [](*hash_keys)
+      if hash_keys.one?
+        Redis::Marshal.load( redis.hget(redis_key, convert_key(hash_keys.first)) )
+      elsif hash_keys.any?
+        values = redis.hmget( redis_key, *hash_keys.map{ |k| convert_key(k) } )
+        values.map{ |v| Redis::Marshal.load(v) }
+      end
+    end
+
+    def []=(hash_key, value)
+      redis.hset( redis_key, convert_key(hash_key), Redis::Marshal.dump(value) )
+    end
+
+    # set only if key doesn't already exist
+    # equivilent to doing `hash[:key] ||= value`, but more efficient
+    def add(hash_key, value)
+      redis.hsetnx( redis_key, convert_key(hash_key), Redis::Marshal.dump(value) )
+    end
+
+    def key=(new_key)
+      new_key = generate_key if new_key.nil?
+      unless @key.nil? || @key == new_key
+        self.class.copy_hash( redis_key, redis_key(new_key) )
+        clear
+      end
+      @key = new_key
+    end
+
+    def namespace=(new_namespace)
+      unless new_namespace == namespace
+        self.class.copy_hash( redis_key, redis_key(key, new_namespace) )
+        clear
+        @namespace = new_namespace
+      end
+    end
+
+    def keys
+      self.class.keys redis_key
+    end
+
+    def key?(hash_key)
+      keys.include?(convert_key(hash_key))
+    end
+    alias_method :include?, :key?
+    alias_method :has_key?, :key?
+    alias_method :member?,  :key?
+
+    def size
+      redis.hlen redis_key
+    end
+    alias_method :count,  :size
+    alias_method :length, :size
+
+    def update(other_hash)
+      writes = []
+      other_hash.each_pair do |hash_key, value|
+        writes << hash_key.to_s
+        writes << Redis::Marshal.dump( value )
+      end
+      redis.hmset(redis_key, *writes)
+    end
+    alias_method :merge,  :update
+    alias_method :merge!, :update
+
+    def delete(hash_key)
+      current_value = self[hash_key]
+      redis.hdel( redis_key, hash_key )
+      current_value
+    end
+
+    def clear
+      redis.del redis_key
+    end
+    alias_method :destroy, :clear
+
+    class << self
+      def keys(redis_key)
+        redis.hkeys redis_key
+      end
+
+      def copy_hash(source_key, dest_key)
+        keys(source_key).each do |k|
+          redis.hset( dest_key, k,
+            redis.hget(source_key, k) )
+        end
+      end
+    end
+  end
+end
+

data/lib/redis/client_helper.rb

@@ -0,0 +1,35 @@
+class Redis
+  class Client
+    def self.default
+      @@default_connection ||= ::Redis.new
+    end
+    def self.default=(connection)
+      @@default_connection = connection
+    end
+  end
+  module ClientHelper
+    def self.included(base)
+      base.send(:extend,  ClassMethods)
+      base.send(:include, InstanceMethods)
+    end
+    module InstanceMethods
+      def redis
+        @redis ||= self.class.redis
+      end
+      def redis=(connection)
+        @redis = connection
+      end
+    end
+    module ClassMethods
+      def redis
+        unless self.class_variable_defined?(:'@@redis')
+          self.class_variable_set(:'@@redis', ::Redis::Client.default)
+        end
+        self.class_variable_get(:'@@redis')
+      end
+      def redis=(connection)
+        self.class_variable_set(:'@@redis', connection)
+      end
+    end
+  end
+end

data/lib/redis/key_helper.rb

@@ -0,0 +1,24 @@
+class Redis
+  module KeyHelper
+
+    def key
+      @key ||= generate_key
+    end
+
+    def generate_key
+      t = Time.now
+      t.strftime('%Y%m%d%H%M%S.') + t.usec.to_s.rjust(6,'0') + '.' + SecureRandom.hex(16)
+    end
+
+    def redis_key(key = nil, namespace = nil)
+      key       ||= self.key
+      namespace ||= self.namespace
+      namespace.nil? ? key : "#{namespace}:#{key}"
+    end
+
+    def convert_key(key)
+      key.to_s
+    end
+  end
+end
+

data/lib/redis/lazy_hash.rb

@@ -0,0 +1,64 @@
+require 'forwardable'
+
+class Redis
+  class LazyHash
+    extend Forwardable
+    def_delegators :@hash, :key, :namespace, :namespace=, :destroy,
+                           :reload, :reload!, :expire
+
+    def initialize(args = nil)
+      @hash = NativeHash.new(args)
+      @loaded = false
+    end
+
+    def method_missing(meth, *args, &block)
+      if @hash.respond_to?(meth)
+        self.class.send(:define_method, meth) do |*args, &block|
+          lazy_load!
+          @hash.send(meth, *args, &block)
+        end
+        send(meth, *args, &block)
+      else
+        super
+      end
+    end
+
+    def inspect
+      lazy_load!
+      @hash.inspect
+    end
+
+    def save
+      @hash.save if loaded?
+    end
+
+    def loaded?
+      @loaded
+    end
+
+    def to_hash
+      self
+    end
+
+    private
+
+      def lazy_load!
+        unless loaded?
+          reload!
+          @hash.retrack!
+          @loaded = true
+        end
+      end
+    
+    class << self
+      def find(args)
+        case args
+        when Hash
+          self.new(args)
+        when String,Symbol
+          self.new(nil=>args)
+        end
+      end
+    end
+  end
+end

data/lib/redis/marshal.rb

@@ -7,11 +7,12 @@ class Redis
       when Fixnum
         value
       else
-        ::Marshal.dump(value)
+        ::Marshal.dump(value) rescue nil
       end
     end
 
     def self.load(value)
+      return nil if value.nil?
       return value unless value.start_with?("\004")
       ::Marshal.load(value) rescue value
     end