redis-objects 0.2.3 → 0.3.0

data/CHANGELOG.rdoc

@@ -0,0 +1,37 @@
+= Changelog for Redis::Objects
+
+== 0.3.0 [Final] (14 April 2010)
+
+* Due to Ruby 1.9 bugs and performance considerations, marshaling of
+data types is now OFF by default.  You must say :marshal => true for
+any objects that you want serialization enabled on. [Nate Wiger]
+
+* Sorted Set class changed slightly due to feedback. You can now get
+an individual element back via @set['item'] since it acts like a Hash.
+
+== 0.2.4 [Final] (9 April 2010)*
+
+* Added sorted set support via Redis::SortedSet [Nate Wiger]
+
+== 0.2.3 [Final] (18 February 2010)*
+
+* Added lock expiration to Redis::Lock [Ben VandenBos]
+
+* Fixed some bugs [Ben VandenBos]
+
+* Added lock tests and test helpers [Ben VandenBos]
+
+== 0.2.2 [Final] (14 December 2009)*
+
+* Added @set.diff(@set2) with "^" and "-" synonyms (oversight). [Nate Wiger]
+
+* Implemented Redis core commands in all data types, such as rename. [Nate Wiger]
+
+* Renamed Redis::Serialize to Redis::Helpers::Serialize to keep Redis:: cleaner. [Nate Wiger]
+
+* More spec coverage. [Nate Wiger]
+
+== 0.2.1 [Final] (27 November 2009)*
+
+* First worthwhile public release, with good spec coverage and functionality. [Nate Wiger]
+

data/README.rdoc

@@ -1,26 +1,31 @@
 = Redis::Objects - Map Redis types directly to Ruby objects
 
-This is *not* an ORM.  People that are wrapping ORM's around Redis are missing
-the point.
+This is *not* an ORM. People that are wrapping ORM’s around Redis are missing the point.
 
-The killer feature of Redis that it allows you to perform atomic operations
-on _individual_ data structures, like counters, lists, and sets. You can then use
-these *with* your existing ActiveRecord/DataMapper/etc models, or in classes that have
-nothing to do with an ORM or even a database.  This gem maps {Redis types}[http://code.google.com/p/redis/wiki/CommandReference]
-to Ruby objects, via a thin layer over Ezra's +redis+ gem.
+The killer feature of Redis is that it allows you to perform _atomic_ operations
+on _individual_ data structures, like counters, lists, and sets.  The *atomic* part is HUGE.
+Using an ORM wrapper that retrieves a "record", updates values, then sends those values back,
+_removes_ the atomicity, cutting the nuts off the major advantage of Redis.  Just use MySQL, k?
+
+This gem provides a Rubyish interface to Redis, by mapping {Redis types}[http://code.google.com/p/redis/wiki/CommandReference]
+to Ruby objects, via a thin layer over Ezra's +redis+ gem.  It offers several advantages
+over the lower-level redis-rb API:
+
+1. Easy to integrate directly with existing ORMs - ActiveRecord, DataMapper, etc.  Add counters to your model!
+2. Complex data structures are automatically Marshaled (if you set :marshal => true)
+3. Integers are returned as integers, rather than '17'
+4. Higher-level types are provided, such as Locks, that wrap multiple calls
 
 This gem originally arose out of a need for high-concurrency atomic operations;
-for a fun rant on the topic, see
-{ATOMICITY}[http://github.com/nateware/redis-objects/blob/master/ATOMICITY.rdoc],
+for a fun rant on the topic, see {An Atomic Rant}[http://nateware.com/2010/02/18/an-atomic-rant],
 or scroll down to "Atomicity" in this README.
 
-There are two ways to use Redis::Objects, either as an +include+ in a model class,
-or by using +new+ with the type of data structure you want to create.
+There are two ways to use Redis::Objects, either as an include in a model class (to
+integrate with ORMs or other classes), or by using new with the type of data structure
+you want to create. 
 
 == Installation
 
-    gem install gemcutter
-    gem tumble
     gem install redis-objects
 
 == Example 1: Model Class Usage
@@ -29,27 +34,30 @@ Using Redis::Objects this way makes it trivial to integrate Redis types with an
 existing ActiveRecord model, DataMapper resource, or other class.  Redis::Objects
 will work with _any_ class that provides an +id+ method that returns a unique
 value.  Redis::Objects will automatically create keys that are unique to
-each object.
+each object, in the format:
+
+    model_name:id:field_name
 
 === Initialization
 
-Redis::Objects needs a handle created by Redis.new.  If you're on Rails,
-config/initializers/redis.rb is a good place for this:
+Redis::Objects needs a handle created by Redis.new.  (If you're on Rails,
+config/initializers/redis.rb is a good place for this.)
 
     require 'redis'
     require 'redis/objects'
     Redis::Objects.redis = Redis.new(:host => 127.0.0.1, :port => 6379)
 
-You can use Redis::Objects with any framework.  There are *no* dependencies on Rails.
-I use it from Sinatra and rake tasks all the time.
+Remember you can use Redis::Objects in any Ruby code.  There are *no* dependencies
+on Rails.  Standalone, Sinatra, Resque - no problem.
 
 === Model Class
 
-Include Redis::Objects in any type of class:
+You can include Redis::Objects in any type of class:
 
     class Team < ActiveRecord::Base
       include Redis::Objects
 
+      lock :trade_players, :expiration => 15  # sec
       counter :hits
       counter :runs
       counter :outs
@@ -82,7 +90,7 @@ Sets work too:
     end
     player = @team.outfielders.detect{|of| of == 'outfielder2'}
 
-And you can do intersections between ORM objects (kinda cool):
+And you can do intersections between objects (kinda cool):
 
     @team1.outfielders | @team2.outfielders   # outfielders on both teams
     @team1.outfielders & @team2.outfielders   # in baseball, should be empty :-)
@@ -101,26 +109,28 @@ Finally, for free, you get a +redis+ method that points directly to a Redis conn
     @team.redis.get('somekey')
     @team.redis.smembers('someset')
 
-You can use the +redis+ handle to directly call any {Redis command}[http://code.google.com/p/redis/wiki/CommandReference]
+You can use the +redis+ handle to directly call any {Redis API command}[http://code.google.com/p/redis/wiki/CommandReference].
 
 == Example 2: Standalone Usage
 
-There is a Ruby object that maps to each Redis type.
+There is a Ruby class that maps to each Redis type, with methods for each
+{Redis API command}[http://code.google.com/p/redis/wiki/CommandReference].
+Note that calling +new+ does not imply it's actually a "new" value - it just
+creates a mapping between that object and the corresponding Redis data structure,
+which may already exist on the redis-server.
 
 === Initialization
 
-Again, Redis::Objects needs a handle to the +redis+ server.  For standalone use, you
+Redis::Objects needs a handle to the +redis+ server.  For standalone use, you
 can either set the $redis global variable:
 
     $redis = Redis.new(:host => 'localhost', :port => 6379)
-    @value = Redis::Value.new('myvalue')
+    @list  = Redis::List.new('mylist')
 
 Or you can pass the Redis handle into the new method for each type:
 
     redis  = Redis.new(:host => 'localhost', :port => 6379)
-    @value = Redis::Value.new('myvalue', redis)
-
-Your choice.
+    @list  = Redis::List.new('mylist', redis)
 
 === Counters
 
@@ -140,6 +150,34 @@ This gem provides a clean way to do atomic blocks as well:
 
 See the section on "Atomicity" for cool uses of atomic counter blocks.
 
+=== Locks
+
+A convenience class that wraps the pattern of {using +setnx+ to perform locking}[http://code.google.com/p/redis/wiki/SetnxCommand].
+
+    require 'redis/lock'
+    @lock = Redis::Lock.new('image_resizing', :expiration => 15, :timeout => 0.1)
+    @lock.lock do
+      # do work
+    end
+
+This can be especially useful if you're running batch jobs spread across multiple hosts.
+
+=== Values
+
+Simple values are easy as well:
+
+    require 'redis/value'
+    @value = Redis::Value.new('value_name')
+    @value.value = 'a'
+    @value.delete
+
+Complex data is no problem with :marshal => true:
+
+    @account = Account.create!(params[:account])
+    @newest  = Redis::Value.new('newest_account', :marshal => true)
+    @newest.value = @account.attributes
+    puts @newest.value['username']
+
 === Lists
 
 Lists work just like Ruby arrays:
@@ -160,14 +198,15 @@ Lists work just like Ruby arrays:
     @list.clear
     # etc
 
-Complex data types are no problem:
+Complex data types are no problem with :marshal => true:
 
+    @list = Redis::List.new('list_name', :marshal => true)
     @list << {:name => "Nate", :city => "San Diego"}
     @list << {:name => "Peter", :city => "Oceanside"}
     @list.each do |el|
       puts "#{el[:name]} lives in #{el[:city]}"
     end
-
+    
 === Sets
 
 Sets work like the Ruby {Set}[http://ruby-doc.org/core/classes/Set.html] class:
@@ -208,8 +247,10 @@ Or store them in Redis:
     @set1.diffstore('diffname', @set2, @set3)
     members = @set1.redis.get('diffname')
 
-And use complex data types too:
+And use complex data types too, with :marshal => true:
 
+    @set1 = Redis::Set.new('set1', :marshal => true)
+    @set2 = Redis::Set.new('set2', :marshal => true)
     @set1 << {:name => "Nate", :city => "San Diego"}
     @set1 << {:name => "Peter", :city => "Oceanside"}
     @set2 << {:name => "Nate", :city => "San Diego"}
@@ -219,26 +260,50 @@ And use complex data types too:
     @set1 - @set2  # Peter
     @set1 | @set2  # all 3 people
 
-=== Values
+=== Sorted Sets
 
-Simple values are easy as well:
+Due to their unique properties, Sorted Sets work like a hybrid between
+a Hash and an Array.  You assign like a Hash, but retrieve like an Array:
 
-    require 'redis/value'
-    @value = Redis::Value.new('value_name')
-    @value.value = 'a'
-    @value.delete
+    require 'redis/sorted_set'
+    @sorted_set = Redis::SortedSet.new('number_of_posts')
+    @sorted_set['Nate']  = 15
+    @sorted_set['Peter'] = 75
+    @sorted_set['Jeff']  = 24
 
-Of course complex data is no problem:
+    # Array access to get sorted order
+    @sorted_set[0..2]           # => ["Nate", "Jeff", "Peter"]
+    @sorted_set[0,2]            # => ["Nate", "Jeff"]
 
-    @account = Account.create!(params[:account])
-    @newest  = Redis::Value.new('newest_account')
-    @newest.value = @account
+    @sorted_set['Peter']        # => 75
+    @sorted_set['Jeff']         # => 24
+    @sorted_set.score('Jeff')   # same thing (24)
+
+    @sorted_set.rank('Peter')   # => 2
+    @sorted_set.rank('Jeff')    # => 1
+
+    @sorted_set.first           # => "Nate"
+    @sorted_set.last            # => "Peter"
+    @sorted_set.revrange(0,2)   # => ["Peter", "Jeff", "Nate"]
+
+    @sorted_set['Newbie'] = 1
+    @sorted_set.members         # => ["Newbie", "Nate", "Jeff", "Peter"]
+
+    @sorted_set.rangebyscore(10, 100, :limit => 2)   # => ["Nate", "Jeff"]
+    @sorted_set.members(:withscores => true)         # => [["Newbie", 1], ["Nate", 16], ["Jeff", 28], ["Peter", 76]]
+
+    # atomic increment
+    @sorted_set.increment('Nate')
+    @sorted_set.incr('Peter')   # shorthand
+    @sorted_set.incr('Jeff', 4)
+
+The other Redis Sorted Set commands are supported as well; see {Sorted Sets API}[http://code.google.com/p/redis/wiki/SortedSets].
 
 == Atomic Counters and Locks
 
 You are probably not handling atomicity correctly in your app.  For a fun rant
 on the topic, see
-{ATOMICITY}[http://github.com/nateware/redis-objects/blob/master/ATOMICITY.rdoc].
+{An Atomic Rant}[http://nateware.com/2010/02/18/an-atomic-rant].
 
 Atomic counters are a good way to handle concurrency:
 

data/lib/redis/counter.rb

@@ -11,10 +11,10 @@ class Redis
     include Redis::Helpers::CoreCommands
 
     attr_reader :key, :options, :redis
-    def initialize(key, redis=$redis, options={})
+    def initialize(key, *args)
       @key = key
-      @redis = redis
-      @options = options
+      @options = args.last.is_a?(Hash) ? args.pop : {}
+      @redis = args.first || $redis
       @options[:start] ||= 0
       @redis.setnx(key, @options[:start]) unless @options[:start] == 0 || @options[:init] === false
     end
@@ -25,6 +25,7 @@ class Redis
     # disconnecting all players).
     def reset(to=options[:start])
       redis.set key, to.to_i
+      true  # hack for redis-rb regression
     end
 
     # Returns the current value of the counter.  Normally just calling the
@@ -42,7 +43,7 @@ class Redis
     # counter will automatically be decremented to its previous value.  This
     # method is aliased as incr() for brevity.
     def increment(by=1, &block)
-      val = redis.incr(key, by).to_i
+      val = redis.incrby(key, by).to_i
       block_given? ? rewindable_block(:decrement, val, &block) : val
     end
     alias_method :incr, :increment
@@ -53,7 +54,7 @@ class Redis
     # counter will automatically be incremented to its previous value.  This
     # method is aliased as incr() for brevity.
     def decrement(by=1, &block)
-      val = redis.decr(key, by).to_i
+      val = redis.decrby(key, by).to_i
       block_given? ? rewindable_block(:increment, val, &block) : val
     end
     alias_method :decr, :decrement

data/lib/redis/helpers/core_commands.rb

@@ -15,7 +15,7 @@ class Redis
       def type
         redis.type key
       end
-      
+
       def rename(name, setkey=true)
         dest = name.is_a?(self.class) ? name.key : name
         ret  = redis.rename key, dest
@@ -41,6 +41,14 @@ class Redis
       def move(dbindex)
         redis.move key, dbindex
       end
+      
+      # See the documentation for SORT: http://code.google.com/p/redis/wiki/SortCommand
+      # TODO
+      # def sort(options)
+      #   args = []
+      #   args += ['sort']
+      #   from_redis redis.sort key
+      # end
     end
   end
 end

data/lib/redis/helpers/serialize.rb

@@ -4,6 +4,7 @@ class Redis
       include Marshal
 
       def to_redis(value)
+        return value unless options[:marshal]
         case value
         when String, Fixnum, Bignum, Float
           value
@@ -13,6 +14,7 @@ class Redis
       end
     
       def from_redis(value)
+        return value unless options[:marshal]
         case value
         when Array
           value.collect{|v| from_redis(v)}

data/lib/redis/list.rb

@@ -12,10 +12,10 @@ class Redis
     include Redis::Helpers::Serialize
 
     attr_reader :key, :options, :redis
-    def initialize(key, redis=$redis, options={})
+    def initialize(key, *args)
       @key = key
-      @redis = redis
-      @options = options
+      @options = args.last.is_a?(Hash) ? args.pop : {}
+      @redis = args.first || $redis
     end
     
     # Works like push.  Can chain together: list << 'a' << 'b'
@@ -62,7 +62,7 @@ class Redis
         at(index)
       end
     end
-    
+
     # Delete the element(s) from the list that match name. If count is specified,
     # only the first-N (if positive) or last-N (if negative) will be removed.
     # Use .del to completely delete the entire key.

data/lib/redis/lock.rb

@@ -10,11 +10,12 @@ class Redis
     class LockTimeout < StandardError; end #:nodoc:
 
     attr_reader :key, :options, :redis
-    def initialize(key, redis=$redis, options={})
+    def initialize(key, *args)
       @key = key
-      @redis = redis
-      @options = options
+      @options = args.last.is_a?(Hash) ? args.pop : {}
+      @redis = args.first || $redis
       @options[:timeout] ||= 5
+      @options[:init] = false if @options[:init].nil? # default :init to false
       @redis.setnx(key, @options[:start]) unless @options[:start] == 0 || @options[:init] === false
     end
 

data/lib/redis/objects/counters.rb

@@ -56,7 +56,7 @@ class Redis
         def increment_counter(name, id=nil, by=1, &block)
           verify_counter_defined!(name, id)
           initialize_counter!(name, id)
-          value = redis.incr(field_key(name, id), by).to_i
+          value = redis.incrby(field_key(name, id), by).to_i
           block_given? ? rewindable_block(:decrement_counter, name, id, value, &block) : value
         end
 
@@ -65,7 +65,7 @@ class Redis
         def decrement_counter(name, id=nil, by=1, &block)
           verify_counter_defined!(name, id)
           initialize_counter!(name, id)
-          value = redis.decr(field_key(name, id), by).to_i
+          value = redis.decrby(field_key(name, id), by).to_i
           block_given? ? rewindable_block(:increment_counter, name, id, value, &block) : value
         end
 
@@ -73,7 +73,8 @@ class Redis
         def reset_counter(name, id=nil, to=nil)
           verify_counter_defined!(name, id)
           to = @redis_objects[name][:start] if to.nil?
-          redis.set(field_key(name, id), to)
+          redis.set(field_key(name, id), to.to_i)
+          true
         end
         
         private

data/lib/redis/objects/locks.rb

@@ -16,7 +16,6 @@ class Redis
         # so it can be used alongside ActiveRecord/DataMapper, etc.
         def lock(name, options={})
           options[:timeout] ||= 5  # seconds
-          options[:init] = false if options[:init].nil? # default :init to false
           @redis_objects[name] = options.merge(:type => :lock)
           if options[:global]
             instance_eval <<-EndMethods
@@ -36,9 +35,6 @@ class Redis
               end
             EndMethods
           end
-
-          
-          
         end
 
         # Obtain a lock, and execute the block synchronously.  Any other code

data/lib/redis/objects/sorted_sets.rb

@@ -0,0 +1,45 @@
+# This is the class loader, for use as "include Redis::Objects::Sets"
+# For the object itself, see "Redis::Set"
+require 'redis/sorted_set'
+class Redis
+  module Objects
+    module SortedSets
+      def self.included(klass)
+        klass.send :include, InstanceMethods
+        klass.extend ClassMethods
+      end
+
+      # Class methods that appear in your class when you include Redis::Objects.
+      module ClassMethods
+        # Define a new list.  It will function like a regular instance
+        # method, so it can be used alongside ActiveRecord, DataMapper, etc.
+        def sorted_set(name, options={})
+          @redis_objects[name] = options.merge(:type => :sorted_set)
+          if options[:global]
+            instance_eval <<-EndMethods
+              def #{name}
+                @#{name} ||= Redis::SortedSet.new(field_key(:#{name}, ''), redis, @redis_objects[:#{name}])
+              end
+            EndMethods
+            class_eval <<-EndMethods
+              def #{name}
+                self.class.#{name}
+              end
+            EndMethods
+          else
+            class_eval <<-EndMethods
+              def #{name}
+                @#{name} ||= Redis::SortedSet.new(field_key(:#{name}), redis, self.class.redis_objects[:#{name}])
+              end
+            EndMethods
+          end
+          
+        end
+      end
+
+      # Instance methods that appear in your class when you include Redis::Objects.
+      module InstanceMethods
+      end
+    end
+  end
+end

data/lib/redis/objects.rb

@@ -6,7 +6,7 @@ class Redis
   # Redis::Objects enables high-performance atomic operations in your app
   # by leveraging the atomic features of the Redis server.  To use Redis::Objects,
   # first include it in any class you want.  (This example uses an ActiveRecord
-  # subclass, but that is *not* required.) Then, use +counter+ and +lock+
+  # subclass, but that is *not* required.) Then, use +counter+, +lock+, +set+, etc
   # to define your primitives:
   #
   #   class Game < ActiveRecord::Base
@@ -14,8 +14,8 @@ class Redis
   #
   #     counter :joined_players
   #     counter :active_players
-  #     set :player_ids
   #     lock :archive_game
+  #     set :player_ids
   #   end
   # 
   # The, you can use these counters both for bookeeping and as atomic actions:
@@ -39,10 +39,11 @@ class Redis
     dir = File.expand_path(__FILE__.sub(/\.rb$/,''))
 
     autoload :Counters, File.join(dir, 'counters')
-    autoload :Values, File.join(dir, 'values')
     autoload :Lists, File.join(dir, 'lists')
-    autoload :Sets, File.join(dir, 'sets')
     autoload :Locks, File.join(dir, 'locks')
+    autoload :Sets, File.join(dir, 'sets')
+    autoload :SortedSets, File.join(dir, 'sorted_sets')
+    autoload :Values, File.join(dir, 'values')
 
     class NotConnected  < StandardError; end
 
@@ -61,10 +62,11 @@ class Redis
         
         # Pull in each object type
         klass.send :include, Redis::Objects::Counters
-        klass.send :include, Redis::Objects::Values
         klass.send :include, Redis::Objects::Lists
-        klass.send :include, Redis::Objects::Sets
         klass.send :include, Redis::Objects::Locks
+        klass.send :include, Redis::Objects::Sets
+        klass.send :include, Redis::Objects::SortedSets
+        klass.send :include, Redis::Objects::Values
       end
     end
 

data/lib/redis/set.rb

@@ -13,10 +13,10 @@ class Redis
     attr_reader :key, :options, :redis
     
     # Create a new Set.
-    def initialize(key, redis=$redis, options={})
+    def initialize(key, *args)
       @key = key
-      @redis = redis
-      @options = options
+      @options = args.last.is_a?(Hash) ? args.pop : {}
+      @redis = args.first || $redis
     end
 
     # Works like add.  Can chain together: list << 'a' << 'b'