redis-objects 0.2.2 → 0.2.3

data/ChangeLog

@@ -1,4 +1,12 @@
 
+*0.2.3 [Final] (18 February 2010)*
+
+* Added lock expiration to Redis::Lock [Ben VandenBos]
+
+* Fixed some bugs [Ben VandenBos]
+
+* Added lock tests and test helpers [Ben VandenBos]
+
 *0.2.2 [Final] (14 December 2009)*
 
 * Added @set.diff(@set2) with "^" and "-" synonyms (oversight). [Nate Wiger]

data/README.rdoc

@@ -23,23 +23,105 @@ or by using +new+ with the type of data structure you want to create.
     gem tumble
     gem install redis-objects
 
-== Example 1: Standalone Usage
+== Example 1: Model Class Usage
+
+Using Redis::Objects this way makes it trivial to integrate Redis types with an
+existing ActiveRecord model, DataMapper resource, or other class.  Redis::Objects
+will work with _any_ class that provides an +id+ method that returns a unique
+value.  Redis::Objects will automatically create keys that are unique to
+each object.
+
+=== Initialization
+
+Redis::Objects needs a handle created by Redis.new.  If you're on Rails,
+config/initializers/redis.rb is a good place for this:
+
+    require 'redis'
+    require 'redis/objects'
+    Redis::Objects.redis = Redis.new(:host => 127.0.0.1, :port => 6379)
+
+You can use Redis::Objects with any framework.  There are *no* dependencies on Rails.
+I use it from Sinatra and rake tasks all the time.
+
+=== Model Class
+
+Include Redis::Objects in any type of class:
+
+    class Team < ActiveRecord::Base
+      include Redis::Objects
+
+      counter :hits
+      counter :runs
+      counter :outs
+      counter :inning, :start => 1
+      list :on_base
+      set :outfielders
+      value :at_bat
+    end
+
+Familiar Ruby array operations Just Work (TM):
+
+    @team = Team.find_by_name('New York Yankees')
+    @team.on_base << 'player1'
+    @team.on_base << 'player2'
+    @team.on_base << 'player3'
+    @team.on_base    # ['player1', 'player2', 'player3']
+    @team.on_base.pop
+    @team.on_base.shift
+    @team.on_base.length  # 1
+    @team.on_base.delete('player2')
+
+Sets work too:
+
+    @team.outfielders << 'outfielder1'
+    @team.outfielders << 'outfielder2'
+    @team.outfielders << 'outfielder1'   # dup ignored
+    @team.outfielders  # ['outfielder1', 'outfielder2']
+    @team.outfielders.each do |player|
+      puts player
+    end
+    player = @team.outfielders.detect{|of| of == 'outfielder2'}
+
+And you can do intersections between ORM objects (kinda cool):
+
+    @team1.outfielders | @team2.outfielders   # outfielders on both teams
+    @team1.outfielders & @team2.outfielders   # in baseball, should be empty :-)
+
+Counters can be atomically incremented/decremented (but not assigned):
+
+    @team.hits.increment  # or incr
+    @team.hits.decrement  # or decr
+    @team.hits.incr(3)    # add 3
+    @team.runs = 4        # exception
+
+Finally, for free, you get a +redis+ method that points directly to a Redis connection:
+
+    Team.redis.get('somekey')
+    @team = Team.new
+    @team.redis.get('somekey')
+    @team.redis.smembers('someset')
+
+You can use the +redis+ handle to directly call any {Redis command}[http://code.google.com/p/redis/wiki/CommandReference]
+
+== Example 2: Standalone Usage
 
 There is a Ruby object that maps to each Redis type.
 
 === Initialization
 
-This gem needs a handle to the +redis+ server.  For standalone use, you can
-either set the $redis global variable to your Redis.new handle:
+Again, Redis::Objects needs a handle to the +redis+ server.  For standalone use, you
+can either set the $redis global variable:
 
     $redis = Redis.new(:host => 'localhost', :port => 6379)
     @value = Redis::Value.new('myvalue')
 
-Or you can pass the Redis handle into the new method:
+Or you can pass the Redis handle into the new method for each type:
 
     redis  = Redis.new(:host => 'localhost', :port => 6379)
     @value = Redis::Value.new('myvalue', redis)
 
+Your choice.
+
 === Counters
 
 Create a new counter. The +counter_name+ is the key stored in Redis.
@@ -111,8 +193,8 @@ You can perform Redis intersections/unions/diffs easily:
     members = @set1 & @set2   # intersection
     members = @set1 | @set2   # union
     members = @set1 + @set2   # union
-    members = @set1 ^ @set2   # union
-    members = @set1 - @set2   # union
+    members = @set1 ^ @set2   # difference
+    members = @set1 - @set2   # difference
     members = @set1.intersection(@set2, @set3)  # multiple
     members = @set1.union(@set2, @set3)         # multiple
     members = @set1.difference(@set2, @set3)    # multiple
@@ -152,80 +234,6 @@ Of course complex data is no problem:
     @newest  = Redis::Value.new('newest_account')
     @newest.value = @account
 
-== Example 2: Model Class Usage
-
-Using Redis::Objects this way makes it trivial to integrate Redis types with an
-existing ActiveRecord model, DataMapper resource, or other class.  Redis::Objects
-will work with _any_ class that provides an +id+ method that returns a unique
-value.  Redis::Objects will automatically create keys that are unique to
-each object.
-
-=== Initialization
-
-If on Rails, config/initializers/redis.rb is a good place for this:
-
-    require 'redis'
-    require 'redis/objects'
-    Redis::Objects.redis = Redis.new(:host => 127.0.0.1, :port => 6379)
-
-=== Model Class
-
-Include Redis::Objects in any type of class:
-
-    class Team < ActiveRecord::Base
-      include Redis::Objects
-      
-      counter :hits
-      counter :runs
-      counter :outs
-      counter :inning, :start => 1
-      list :on_base
-      set :outfielders
-      value :at_bat
-    end
-
-Familiar Ruby array operations Just Work (TM):
-
-    @team = Team.find_by_name('New York Yankees')
-    @team.on_base << 'player1'
-    @team.on_base << 'player2'
-    @team.on_base << 'player3'
-    @team.on_base    # ['player1', 'player2']
-    @team.on_base.pop
-    @team.on_base.shift
-    @team.on_base.length  # 1
-    @team.on_base.delete('player3')
-
-Sets work too:
-    
-    @team.outfielders << 'outfielder1' << 'outfielder1'
-    @team.outfielders << 'outfielder2'
-    @team.outfielders  # ['outfielder1', 'outfielder2']
-    @team.outfielders.each do |player|
-      puts player
-    end
-    player = @team.outfielders.detect{|of| of == 'outfielder2'}
-
-And you can do intersections between ORM objects (kinda cool):
-
-    @team1.outfielders | @team2.outfielders   # all outfielders
-    @team1.outfielders & @team2.outfielders   # should be empty
-
-Counters can be atomically incremented/decremented (but not assigned):
-
-    @team.hits.increment  # or incr
-    @team.hits.decrement  # or decr
-    @team.hits.incr(3)    # add 3
-    @team.runs = 4        # exception
-
-Finally, for free, you get a +redis+ handle usable in your class that
-points directly to a Redis API object:
-
-    @team.redis.get('somekey')
-    @team.redis.smembers('someset')
-
-You can use the +redis+ handle to directly call any {Redis command}[http://code.google.com/p/redis/wiki/CommandReference]
-
 == Atomic Counters and Locks
 
 You are probably not handling atomicity correctly in your app.  For a fun rant
@@ -277,6 +285,10 @@ Class-level atomic block (may save a DB fetch depending on your app):
 
 Locks with Redis. On completion or exception the lock is released:
 
+    class Team < ActiveRecord::Base
+      lock :reorder # declare a lock
+    end
+
     @team.reorder_lock.lock do
       @team.reorder_all_players
     end
@@ -287,9 +299,19 @@ Class-level lock (same concept)
       Team.reorder_all_players(team_id)
     end
 
+Lock expiration.  Sometimes you want to make sure your locks are cleaned up should
+the unthinkable happen (server failure).  You can set lock expirations to handle
+this.  Expired locks are released by the next process to attempt lock.  Just
+make sure you expiration value is sufficiently large compared to your expected
+lock time.
+
+    class Team < ActiveRecord::Base
+      lock :reorder, :expiration => 15.minutes
+    end
+
 
 == Author
 
-Copyright (c) 2009 {Nate Wiger}[http://nate.wiger.org].  All Rights Reserved.
+Copyright (c) 2009-2010 {Nate Wiger}[http://nate.wiger.org].  All Rights Reserved.
 Released under the {Artistic License}[http://www.opensource.org/licenses/artistic-license-2.0.php].
 

data/lib/redis/lock.rb

@@ -31,17 +31,53 @@ class Redis
     def lock(&block)
       start = Time.now
       gotit = false
+      expiration = nil
       while Time.now - start < @options[:timeout]
-        gotit = redis.setnx(key, 1)
+        expiration = generate_expiration
+        # Use the expiration as the value of the lock.
+        gotit = redis.setnx(key, expiration)
         break if gotit
+
+        # Lock is being held.  Now check to see if it's expired (if we're using
+        # lock expiration).
+        # See "Handling Deadlocks" section on http://code.google.com/p/redis/wiki/SetnxCommand
+        if !@options[:expiration].nil?
+          old_expiration = redis.get(key).to_f
+
+          if old_expiration < Time.now.to_f
+            # If it's expired, use GETSET to update it.
+            expiration = generate_expiration
+            old_expiration = redis.getset(key, expiration).to_f
+
+            # Since GETSET returns the old value of the lock, if the old expiration
+            # is still in the past, we know no one else has expired the locked
+            # and we now have it.
+            if old_expiration < Time.now.to_f
+              gotit = true
+              break
+            end
+          end
+        end
+
         sleep 0.1
       end
       raise LockTimeout, "Timeout on lock #{key} exceeded #{@options[:timeout]} sec" unless gotit
       begin
         yield
       ensure
-        redis.del(key)
+        # We need to be careful when cleaning up the lock key.  If we took a really long
+        # time for some reason, and the lock expired, someone else may have it, and
+        # it's not safe for us to remove it.  Check how much time has passed since we
+        # wrote the lock key and only delete it if it hasn't expired (or we're not using
+        # lock expiration)
+        if @options[:expiration].nil? || expiration > Time.now.to_f
+          redis.del(key)
+        end
       end
     end
+
+    def generate_expiration
+      @options[:expiration].nil? ? 1 : (Time.now + @options[:expiration].to_f + 1).to_f
+    end
   end
 end

data/lib/redis/objects/locks.rb

@@ -16,6 +16,7 @@ class Redis
         # so it can be used alongside ActiveRecord/DataMapper, etc.
         def lock(name, options={})
           options[:timeout] ||= 5  # seconds
+          options[:init] = false if options[:init].nil? # default :init to false
           @redis_objects[name] = options.merge(:type => :lock)
           if options[:global]
             instance_eval <<-EndMethods
@@ -47,7 +48,7 @@ class Redis
           verify_lock_defined!(name)
           raise ArgumentError, "Missing block to #{self.name}.obtain_lock" unless block_given?
           lock_name = field_key("#{name}_lock", id)
-          Redis::Lock.new(redis, lock_name, self.class.redis_objects[name]).lock(&block)
+          Redis::Lock.new(lock_name, redis, self.redis_objects[name]).lock(&block)
         end
 
         # Clear the lock.  Use with care - usually only in an Admin page to clear

data/spec/redis_objects_instance_spec.rb

@@ -5,6 +5,7 @@ require 'redis/counter'
 require 'redis/list'
 require 'redis/set'
 require 'redis/value'
+require 'redis/lock'
 
 describe Redis::Value do
   before :all do
@@ -329,4 +330,91 @@ describe Redis::Counter do
   after :all do
     @counter.delete
   end
+end
+
+describe Redis::Lock do
+
+  before :each do
+    $redis.flushall
+  end
+
+  it "should set the value to the expiration" do
+    start = Time.now
+    expiry = 15
+    lock = Redis::Lock.new(:test_lock, $redis, :expiration => expiry, :init => false)
+    lock.lock do
+      expiration = $redis.get("test_lock").to_f
+
+      # The expiration stored in redis should be 15 seconds from when we started
+      # or a little more
+      expiration.should be_close((start + expiry).to_f, 2.0)
+    end
+
+    # key should have been cleaned up
+    $redis.get("test_lock").should be_nil
+  end
+
+  it "should set value to 1 when no expiration is set" do
+    lock = Redis::Lock.new(:test_lock, $redis, :init => false)
+    lock.lock do
+      $redis.get('test_lock').should == '1'
+    end
+
+    # key should have been cleaned up
+    $redis.get("test_lock").should be_nil
+  end
+
+  it "should let lock be gettable when lock is expired" do
+    expiry = 15
+    lock = Redis::Lock.new(:test_lock, $redis, :expiration => expiry, :timeout => 0.1, :init => false)
+
+    # create a fake lock in the past
+    $redis.set("test_lock", Time.now-(expiry + 60))
+
+    gotit = false
+    lock.lock do
+      gotit = true
+    end
+
+    # should get the lock because it has expired
+    gotit.should be_true
+    $redis.get("test_lock").should be_nil
+  end
+
+  it "should not let non-expired locks be gettable" do
+    expiry = 15
+    lock = Redis::Lock.new(:test_lock, $redis, :expiration => expiry, :timeout => 0.1, :init => false)
+
+    # create a fake lock
+    $redis.set("test_lock", (Time.now + expiry).to_f)
+
+    gotit = false
+    error = nil
+    begin
+      lock.lock do
+        gotit = true
+      end
+    rescue => error
+    end
+
+    error.should be_kind_of(Redis::Lock::LockTimeout)
+
+    # should not have the lock
+    gotit.should_not be_true
+
+    # lock value should still be set
+    $redis.get("test_lock").should_not be_nil
+  end
+
+  it "should not remove the key if lock is held past expiration" do
+    lock = Redis::Lock.new(:test_lock, $redis, :expiration => 0.0, :init => false)
+
+    lock.lock do
+      sleep 1.1
+    end
+
+    # lock value should still be set since the lock was held for more than the expiry
+    $redis.get("test_lock").should_not be_nil
+  end
+
 end

data/spec/redis_objects_model_spec.rb

@@ -282,6 +282,19 @@ describe Redis::Objects do
     end
     error.should be_kind_of(NoMethodError)
   end
+
+  it "should support obtain_lock as a class method" do
+    error = nil
+    begin
+      Roster.obtain_lock(:resort, 2) do
+        Roster.redis.get("roster:2:resort_lock").should_not be_nil
+      end
+    rescue => error
+    end
+
+    error.should be_nil
+    Roster.redis.get("roster:2:resort_lock").should be_nil
+  end
   
   it "should handle simple values" do
     @roster.starting_pitcher.should == nil
@@ -635,4 +648,5 @@ describe Redis::Objects do
     error.should_not be_nil
     error.should be_kind_of(Redis::Lock::LockTimeout)
   end
+
 end

metadata

@@ -1,7 +1,7 @@
 --- !ruby/object:Gem::Specification 
 name: redis-objects
 version: !ruby/object:Gem::Version 
-  version: 0.2.2
+  version: 0.2.3
 platform: ruby
 authors: 
 - Nate Wiger
@@ -9,7 +9,7 @@ autorequire:
 bindir: bin
 cert_chain: []
 
-date: 2009-12-14 00:00:00 -08:00
+date: 2010-02-18 00:00:00 -08:00
 default_executable: 
 dependencies: 
 - !ruby/object:Gem::Dependency