redis_ha 0.1.0 → 0.1.1

data/README.md

@@ -1,17 +1,25 @@
 RedisHA
 =======
 
-A redis client that runs commands on multiple servers in parallel 
-without blocking if one of them is down.
+RedisHA includes:
+
++ A redis client that runs commands on multiple servers in parallel 
+and handles failure gracefully 
+
++ A few highly available data structures / CRDTS (counter, set, hashmap)
+
+
+### Rationale
 
 I used this to implement a highly available session store on top of
-redis; it writes and reads the data to multiple instances and merges 
-the responses after every read. This approach is negligibly slower 
-than writing to a single server since RedisHA uses asynchronous I/O 
-and is much more robust than complex server-side redis failover solutions
-(sentinel, pacemaker, etcetera).
+redis; it writes and reads to multiple servers and merges the responses 
+after every read. 
 
-The gem includes three basic CRDTs (set, hashmap and counter).
+This is negligibly slower than writing to a single server since RedisHA 
+uses asynchronous I/O, but it is more resilient than a complex server-side
+redis failover solution (sentinel, pacemaker, etcetera): you can `kill -9`
+any server at any time and continue to read and write as long as at least
+one server is healthy.
 
 [1] _DeCandia, Hastorun et al_ (2007). [Dynamo: Amazon’s Highly Available Key-value Store](http://www.read.seas.harvard.edu/~kohler/class/cs239-w08/decandia07dynamo.pd) (SOSP 2007)
 
@@ -19,7 +27,7 @@ The gem includes three basic CRDTs (set, hashmap and counter).
 Usage
 -----
 
-Create a RedisHA::ConnectionPool (connect does not block):
+Create a RedisHA::ConnectionPool (`connect` does not block):
 
 ```ruby
 pool = RedisHA::ConnectionPool.new
@@ -37,9 +45,20 @@ Execute a command in parallel:
 => ["PONG", "PONG", "PONG"]
 
 >> pool.setnx "fnord", 1
-=> [1,1,1]
+=> [1, 1, 1]
 ```
 
+Execute a command in parallel when server #2 is down:
+
+```ruby
+>> pool.ping
+=> ["PONG", nil, "PONG"]
+
+>> pool.setnx "fnord", 1
+=> [1, nil, 1]
+```
+
+
 RedisHA::Counter (INCR/DECR/SET/GET)
 
 ```ruby
@@ -82,28 +101,60 @@ RedisHA::Set (ADD/REM/GET)
 => [:fnord]
 ```
 
+
+
+Installation
+------------
+
+    gem install redis_ha
+
+or in your Gemfile:
+
+    gem 'redis_ha', '>= 0.1'
+
+
 Timeouts
 --------
 
-here be dragons
+RedisHA implements two timeouts per connection: A `read_timeout` and a `retry_timeout`
 
+When a server takes longer than read_timeout seconds to respond to a request it is 
+considered down. Once a server is down it is excluded from subsequent requests for the 
+given retry_timeout. 
 
-Caveats
---------
+That means if one server is down, one request will take at least read_timeout seconds
+to complete every retry_timeout seconds.
 
--> delete / decrement is not safe
+The defaults are 500ms for read and 10s for the retry. If you are only using fast redis
+operations you should set the read_timeout to 100ms or lower.
 
+```ruby
+pool = RedisHA::ConnectionPool.new
+pool.retry_timeout = 10
+pool.read_timeout = 0.1
+```
 
 
+Merge Strategies
+----------------
 
-Installation
-------------
+The default merge strategy for `RedisHA::Set` favors addtions over deletions (a deleted
+element might re-appear in a set if a server goes down and comes back up with an
+old / inconsistent state, but a element can never be lost from a set as long as at least
+one server is healthy)
 
-    gem install redis_ha
+The default merge strategy for `RedisHA::Counter` favor increments over decrements (a
+counters value might be greater than the real value in some conditions but it can never
+be less than the real value)
 
-or in your Gemfile:
+You can define your own merge strategy:
+
+```ruby
+>> ctr = RedisHA::Counter.new(pool, "my-counter")  
 
-    gem 'redis_ha', '~> 0.3'
+# select the smallest value when merging counter responses 
+>> ctr.merge_strategy = lambda{ |values| vales.map(&:to_i).min }
+```
 
 
 License

data/lib/redis_ha/crdt/set.rb

@@ -2,7 +2,7 @@ class RedisHA::Set < RedisHA::Base
 
   # this lambda defines how the individual response hashes are merged
   # the default is set union
-  DEFAULT_MERGE_STRATEGY = ->(v) { v.inject(&:+).uniq }
+  DEFAULT_MERGE_STRATEGY = ->(v) { v.inject(&:|) }
 
   def add(*items)
     pool.sadd(@key, *items)

data/lib/redis_ha/protocol.rb

@@ -8,15 +8,31 @@ class RedisHA::Protocol
 
   def self.peek?(buf)
     if ["+", ":", "-"].include?(buf[0])
-      buf[-2..-1] == "\r\n"
-    elsif buf[0] == "$"
+      !!buf.index("\r\n")
+
+    elsif ["$", "*"].include?(buf[0])
       offset = buf.index("\r\n").to_i
       return false if offset == 0
       length = buf[1..offset].to_i
       return true if length == -1
+      offset += 2
+
+      if buf[0] == "*"
+        multi = length
+        length.times do |ind|
+          if buf[offset+1..offset+2] == "-1"
+            offset += 5
+          elsif /^\$(?<len>[0-9]+)\r\n/ =~ buf[offset..-1]
+            length = len.to_i
+            offset += len.length + 3
+            offset += length + 2 if ind < multi - 1
+          else
+            return false
+          end
+        end
+      end
+
       buf.size >= (length + offset + 2)
-    elsif buf[0] == "*"
-      true
     end
   end
 
@@ -27,10 +43,20 @@ class RedisHA::Protocol
       when ":" then buf[1..-3].to_i
 
       when "$"
-         buf.sub(/.*\r\n/,"")[0...-2] if buf[1..2] != "-1"
+        if buf[1..2] == "-1"
+          buf.replace(buf[5..-1] || "")
+          nil
+        else
+          len = buf.match(/^\$([-0-9]+)\r\n/)[1]
+          ret = buf[len.length+3..len.length+len.to_i+2]
+          buf.replace(buf[len.to_i+len.length+5..-1] || "")
+          ret
+        end
 
       when "*"
-        RuntimeError.new("multi bulk replies are not supported")
+        cnt = buf.match(/^\*([0-9]+)\r\n/)[1]
+        buf = buf[cnt.length+3..-1]
+        cnt.to_i.times.map { parse(buf) }
 
     end
   end

data/lib/test.rb

@@ -23,6 +23,10 @@ map = RedisHA::HashMap.new(pool, "fnordmap")
 set = RedisHA::Set.new(pool, "fnordset")
 ctr = RedisHA::Counter.new(pool, "fnordctr")
 
+#set.add(:fnord, :bar, :fubar, :blubb)
+#puts pool.smembers("fnordset").inspect
+#puts set.get.inspect
+
 Ripl.start :binding => binding
 exit
 

data/redis_ha.gemspec

@@ -3,7 +3,7 @@ $:.push File.expand_path("../lib", __FILE__)
 
 Gem::Specification.new do |s|
   s.name        = "redis_ha"
-  s.version     = "0.1.0"
+  s.version     = "0.1.1"
   s.date        = Date.today.to_s
   s.platform    = Gem::Platform::RUBY
   s.authors     = ["Paul Asmuth"]

metadata

@@ -1,7 +1,7 @@
 --- !ruby/object:Gem::Specification
 name: redis_ha
 version: !ruby/object:Gem::Version
-  version: 0.1.0
+  version: 0.1.1
   prerelease: 
 platform: ruby
 authors:
@@ -9,7 +9,7 @@ authors:
 autorequire: 
 bindir: bin
 cert_chain: []
-date: 2012-12-22 00:00:00.000000000 Z
+date: 2012-12-23 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: redis