raidis 0.1.0

data/README.md

@@ -0,0 +1,69 @@
+# Raid-is
+
+Raidis is yet another failover solution for Redis.
+
+## Why not use RedisFailover?
+
+* Firstly, because RedisFailover may fail itself (it depends on Zookeeper, remember?), while the Redis server is perfectly healthy
+* Seondly, RedisFailover is utterly incompatible with the thread'n'fork chaos introduced by Resque workers
+* Thirdly, if you have but one single application not using Ruby, you'll have to come up with a _global_ failover solution anyway
+
+## How do I get started?
+
+Raidis knows where the redis master is by checking the file `/etc/redis_master` and there looking for content such as `127.0.0.1:6379` (you can omit the port if you like to). Note that you can use a RedisFailover daemon to update that file if you wish. At any rate, you won't need to use `Redis.new` or `RedisFailover.new` anymore. You'll find your ready-to-go redis in `Raidis.redis` with zero-configuration in your application.
+
+```bash
+# Bash
+echo "127.0.0.1:6379" > /etc/redis_master
+```
+
+```ruby
+# Ruby application
+require 'raidis'
+Raidis.redis.get('some_key')
+
+Raidis.redis.class # => Redis::Namespace
+```
+
+# How does it work?
+
+Whenever you call `Raidis.redis`, the connectivity to the remote redis server is monitored. If connectivity problems occur, or you're trying to make write-calls to a Redis slave, Raidis will immediately reload the `/etc/redis_master` file and try to perform the call to Redis again (hoping to have connected to a working redis server this time). If that second attempt failed (or you set `config.retries` to `0`) a `Raidis::ConnectionError` is raised. 
+
+You should not have too many retries, because you don't want the end user's browser to hang and wait too long. That's where the `#available?` feature comes in. As soon as one of those connection errors occurs, the global variable `Raidis.available?` turns from `true` to `false` and any further damage can be mitigated, by simply not making any further calls to redis. You should inform your end-users about the outage in a friendly manner. E.g. like so:
+
+```ruby
+if Raidis.available?
+   counter = Raidis.redis.get('visits')
+else
+   counter = "Sorry, the visits counter is not available right now."
+end
+```
+
+Note that it is one of the design goals of this gem that there are no performance penalties when using `.available?`.
+
+After 15 seconds (or whichever timeout you configure), `Raidis.available?` turns to `true` again automatically and the file `/etc/redis_master` is read again in order to find the remote redis server. If you wish, you may use `Raidis.reconnect!` to evoke the end of that unavailability period manually.
+
+# Connection Pools
+
+If you need to maintain a ConnectionPool with multiple connections to Redis (e.g. when you use [Sidekiq](https://github.com/mperham/sidekiq/issues/794)), you may use the something like `ConnectionPool.new { Raidis.redis! }` (note the exclamation mark) to populate your pool. `.raidis!` returns a new connection each time you call it. Note that `Raidis.available?` is a global variable, so if any of the pools fail, Raidis will go into unavailable mode as described above.
+
+## Configuration
+
+Example:
+
+```ruby
+Raidis.configure do |config|
+
+  config.logger          = Rails.logger                # default is Rails.logger (if defined) otherwise: Logger.new(STDOUT)
+  config.redis_namespace = :myapp                      # default is nil
+  config.redis_db        = (Rails.env.test? ? 1 : 0)   # default is whatever Redis.new has as default
+  config.redis_timeout   = 3  # seconds                # default is whatever Redis.new has as default
+
+  config.retries = 3  # times                          # default is 1
+  config.unavailability_timeout = 60  # seconds        # default is 15 seconds
+  config.info_file_path         = '/opt/redis_server'  # default is '/etc/redis_master'
+
+  # You can override the info_file content if you like
+  config.master = '127.0.0.1'                          # if omitted, the content of /etc/redis_master is used
+end
+```

data/lib/raidis.rb

@@ -0,0 +1,51 @@
+require 'redis/namespace'
+require 'timeout'
+
+require 'raidis/availability'
+require 'raidis/configuration'
+require 'raidis/redis_wrapper'
+
+module Raidis
+
+  ConnectionError = Class.new(RuntimeError)
+
+  extend self
+  extend Availability
+
+  # Public: The singleton Redis connection object.
+  #
+  # Returns a RedisWrapper instance.
+  #
+  def redis
+    return @redis if @redis
+    @redis = redis!
+    connected?
+    @redis
+  end
+
+  # Public: Updates the #available? flag by actually testing the Redis connection.
+  #
+  # Returns true or false.
+  #
+  def connected?
+    return unavailable! unless @redis
+    @redis.setex(:raidis, 1, :rocks) && available!
+  rescue Raidis::ConnectionError
+    unavailable!
+  end
+
+  # Public: Evokes a fresh lookup of the Redis server endpoint.
+  #
+  def reconnect!
+    @redis = nil
+    redis
+  end
+
+  # Public: Creates a brand-new failsafe-wrapped connection to Redis.
+  # This is ONLY useful if you need to maintain your own ConnectionPool.
+  # See https://github.com/mperham/sidekiq/issues/794
+  #
+  def redis!
+    RedisWrapper.new
+  end
+end

data/lib/raidis/availability.rb

@@ -0,0 +1,34 @@
+module Raidis
+  module Availability
+
+    def available?
+      if unavailability_age_in_seconds >= config.unavailability_timeout
+        available!
+      else
+        !!@available
+      end
+    end
+
+    def available!
+      checking_availability
+      @available = true
+    end
+
+    def unavailable!
+      checking_availability
+      @available = false
+    end
+
+    private
+
+    def checking_availability
+      @last_availability_check = Time.now.to_i
+    end
+
+    def unavailability_age_in_seconds
+      return 0 unless @last_availability_check
+      Time.now.to_i - @last_availability_check.to_i
+    end
+
+  end
+end

data/lib/raidis/configuration.rb

@@ -0,0 +1,89 @@
+module Raidis
+  class Configuration
+
+    InfoFilePathNotFound = Class.new(RuntimeError)
+
+    class Master
+      attr_accessor :endpoint
+      attr_writer :port
+
+      def port
+        @port ||= 6379
+      end
+    end
+
+    attr_accessor :redis_namespace, :redis_timeout, :redis_db
+    attr_writer :logger, :unavailability_timeout, :master, :retries
+
+    def logger
+      @logger ||= begin
+        if defined?(Rails)
+          Rails.logger
+        else
+          Logger.new(STDOUT)
+        end
+      end
+    end
+
+    def info_file_path
+      @info_file_path ||= Pathname.new('/etc/redis_master')
+    end
+
+    def info_file_path=(path)
+      Pathname.new path
+    end
+
+    def unavailability_timeout
+      @unavailability_timeout ||= 15  # seconds
+    end
+
+    def retries
+      @retries ||= 1
+    end
+
+    def master
+      unless @master
+        unless info_file_path.exist?
+          Trouble.notify(InfoFilePathNotFound.new, code: :info_file_not_found, message: 'Raidis could not find the redis master info file', location: info_file_path) if defined?(Trouble)
+          return
+        end
+
+        unless info_file_path.readable?
+          Trouble.notify(InfoFilePathNotFound.new, code: :info_file_not_readable, message: 'The redis master info file exists but is not readable for Raidis', location: info_file_path) if defined?(Trouble)
+          return
+        end
+      end
+
+      content = @master || info_file_path.read
+      server = Master.new
+      server.endpoint, server.port = content.strip.to_s.split(':')
+
+      unless server.endpoint
+        if @master
+          Trouble.notify(InfoFilePathNotFound.new, code: :invalid_master, message: 'Raidis does not understand the config.master value you provided', value: config.master.inspect) if defined?(Trouble)
+        else
+          Trouble.notify(InfoFilePathNotFound.new, code: :invalid_info_file_content, message: 'Raidis found the redis master info file, but there was no valid endpoint in it', location: info_file_path, content: info_file_path.read.inspect) if defined?(Trouble)
+        end
+        return
+      end
+      server
+    end
+  end
+end
+
+module Raidis
+  extend self
+
+  def config
+    @config ||= Configuration.new
+  end
+
+  def configure(&block)
+    yield config
+  end
+
+  def reset!
+    config = nil
+    reconnect!
+  end
+end

data/lib/raidis/redis_wrapper.rb

@@ -0,0 +1,104 @@
+module Raidis
+  class RedisWrapper
+
+    # Public: Proxies everything to the Redis backend.
+    #
+    # Returns whatever the backend returns.
+    # Raises Raidis::ConnectionError if there is a connection problem.
+    #
+    def method_missing(method, *args, &block)
+      raise(Raidis::ConnectionError, 'No Redis backend found.') unless redis
+      reloading_connection do
+        observing_connection { redis.send(method, *args, &block) }
+      end
+    end
+
+    # Internal: If a Raidis::ConnectionError is detected during the execution of the block,
+    # try to reconnect to Redis and try again. Updates the availability state.
+    #
+    # Returns whatever the block returns.
+    # Raises Raidis::ConnectionError if the connection problem persists even after the retries.
+    #
+    def reloading_connection(&block)
+      tries ||= config.retries
+      result = block.call
+    rescue Raidis::ConnectionError => exception
+      # Try again a couple of times.
+      if (tries -= 1) >= 0
+        reconnect!
+        retry
+      end
+      # Giving up.
+      Raidis.unavailable!
+      raise exception
+    else
+      # No exception was raised, reaffirming the availability.
+      Raidis.available!
+      result
+    end
+
+    # Internal: Raises a Raidis::ConnectionError if there are connection-related problems during the execution of the block.
+    # More specifically, if the connection is lost or a write is performed against a slave, the Exception will be raised.
+    #
+    def observing_connection(&block)
+      yield
+
+    rescue *connection_errors => exception
+      Trouble.notify(exception, code: :lost_connection, message: 'Raidis lost connection to the Redis server.', client: redis.inspect) if defined?(Trouble)
+      raise Raidis::ConnectionError, exception
+
+    rescue Redis::CommandError => exception
+      if exception.message.to_s.split.first == 'READONLY'
+        Trouble.notify(exception, code: :readonly, message: 'Raidis detected an illegal write against a Redis slave.', client: redis.inspect) if defined?(Trouble)
+        raise Raidis::ConnectionError, exception
+      else
+        # Passing through Exceptions unrelated to the Connection. E.g. Redis::CommandError.
+        raise exception
+      end
+    end
+
+    # Internal: A list of known connection-related Exceptions the backend may raise.
+    #
+    def connection_errors
+      [
+        Redis::BaseConnectionError,
+        IOError,
+        Timeout::Error,
+        Errno::EADDRNOTAVAIL,
+        Errno::EAGAIN,
+        Errno::EBADF,
+        Errno::ECONNABORTED,
+        Errno::ECONNREFUSED,
+        Errno::ECONNRESET,
+        Errno::EHOSTUNREACH,
+        Errno::EINVAL,
+        Errno::ENETUNREACH,
+        Errno::EPIPE,
+      ]
+    end
+
+    def reconnect!
+      @redis = nil
+    end
+
+    # Internal: Establishes a brand-new, raw connection to Redis.
+    #
+    # Returns a Redis::Namespace instance or nil if we don't know where the Redis server is.
+    #
+    def redis
+      @redis ||= redis!
+    end
+
+    def redis!
+      return unless master = config.master
+      raw_redis = Redis.new db: config.redis_db, host: master.endpoint, port: master.port, timeout: config.redis_timeout
+      Redis::Namespace.new config.redis_namespace, redis: raw_redis
+    end
+
+    # Internal: Convenience wrapper
+    #
+    def config
+      Raidis.config
+    end
+  end
+end

data/spec/lib/raidis_spec.rb

@@ -0,0 +1,162 @@
+require 'spec_helper'
+
+# This is an integration test that requires a Redis server.
+
+describe Raidis do
+
+  let(:raidis)      { Raidis }
+  let(:redis)       { raidis.redis }
+
+  context 'when connected' do
+    context 'to a Redis master' do
+      before do
+        redis.slaveof :no, :one
+      end
+
+      describe '.redis' do
+        it 'is is a RedisWrapper' do
+          redis.should be_instance_of Raidis::RedisWrapper
+        end
+
+        it 'lets through command errors unrelated to the connection' do
+          expect { redis.setex(:invalid, -1, :parameters) }.to raise_error(Redis::CommandError)
+        end
+      end
+
+      describe '.connected?' do
+        it 'is true' do
+          raidis.should be_connected
+        end
+      end
+
+      describe '.available?' do
+        it 'is true' do
+          raidis.should be_available
+        end
+      end
+    end
+
+    context 'to a Redis slave' do
+      before do
+        redis.slaveof '127.0.0.1', 12345
+        raidis.reconnect!
+      end
+
+      after do
+        redis.slaveof :no, :one
+      end
+
+      describe '.redis' do
+        it 'detects illegal writes to a slave' do
+          Trouble.should_receive(:notify) do |exception, metadata|
+            exception.should be_instance_of Redis::CommandError
+            metadata[:code].should == :readonly
+          end
+          expect { redis.setex(:writing, 1, :against_a_slave) }.to raise_error(Raidis::ConnectionError)
+        end
+
+        it 'is fine with read-only commands' do
+          redis.get(:some_key).should be_nil
+        end
+
+        it 'lets through command errors unrelated to the connection' do
+          expect { redis.lrange(:invalid, :command, :arguments) }.to raise_error(Redis::CommandError)
+        end
+      end
+
+      describe '.connected?' do
+        it 'is false' do
+          raidis.should_not be_connected
+        end
+      end
+
+      describe '.available?' do
+        it 'is false' do
+          raidis.should_not be_available
+        end
+      end
+    end
+  end
+
+  context 'when not connected' do
+    before do
+      Raidis.config.master = '127.0.0.1:0'
+      Raidis.reconnect!
+    end
+
+    context 'unavailability timeout' do
+      before do
+        Raidis.config.unavailability_timeout = 5  # Seconds
+      end
+
+      describe '.available?' do
+        it 'becomes available again after the unavailability_timeout' do
+          raidis.should_not be_available
+          Timecop.travel Time.now + 4
+          raidis.should_not be_available
+          Timecop.travel Time.now + 1
+          raidis.should be_available
+        end
+      end
+    end
+
+    context 'because the Network is unreachable' do
+      before do
+        Raidis.config.master = '192.0.2.1'  # RFC 5737
+        Raidis.reconnect!
+      end
+
+      describe '.redis' do
+        it 'detects that there is no connection' do
+          Trouble.should_receive(:notify) do |exception, metadata|
+            exception.should be_instance_of Redis::CannotConnectError
+            metadata[:code].should == :lost_connection
+          end
+          expect { redis.get(:some_key) }.to raise_error(Raidis::ConnectionError)
+        end
+      end
+
+      describe '.connected?' do
+        it 'is false' do
+          raidis.should_not be_connected
+        end
+      end
+
+      describe '.available?' do
+        it 'is false' do
+          raidis.should_not be_available
+        end
+      end
+    end
+
+    context 'because of a wrong port' do
+      before do
+        Raidis.config.master = '127.0.0.1:80'
+        Raidis.reconnect!
+      end
+
+      describe '.redis' do
+        it 'detects that there is no connection' do
+          Trouble.should_receive(:notify) do |exception, metadata|
+            exception.should be_instance_of Redis::CannotConnectError
+            metadata[:code].should == :lost_connection
+          end
+          expect { redis.get(:some_key) }.to raise_error(Raidis::ConnectionError)
+        end
+      end
+
+      describe '.connected?' do
+        it 'is false' do
+          raidis.should_not be_connected
+        end
+      end
+
+      describe '.available?' do
+        it 'is false' do
+          raidis.should_not be_available
+        end
+      end
+    end
+  end
+
+end

data/spec/lib/redis_wrapper_spec.rb

@@ -0,0 +1,88 @@
+require 'spec_helper'
+
+class ShakyRedis
+  def initialize(failures = 3)
+    @failures = 3
+  end
+
+  def perform
+    @calls ||= 0
+    if @calls < @failures
+      @calls += 1
+      raise Errno::ECONNREFUSED
+    end
+    :shaky_result
+  end
+
+  def method_missing(method, *args, &block)
+    perform
+  end
+end
+
+describe Raidis::RedisWrapper do
+
+  let(:config)        { mock(:config, retries: 3) }
+  let(:backend)       { mock(:backend) }
+  let(:shaky_backend) { ShakyRedis.new }
+  let(:wrapper)       { Raidis::RedisWrapper.new }
+
+  before do
+    wrapper.stub!(:config).and_return config
+  end
+
+  describe '#method_missing' do
+    context 'with a stable connection' do
+      before do
+        wrapper.stub!(:redis!).and_return backend
+      end
+
+      it 'proxies everything to the backend' do
+        backend.should_receive(:any_redis_command).with(:some_key).and_return 'value'
+        wrapper.any_redis_command(:some_key).should == 'value'
+      end
+
+      it 'passes on Exceptions from the backend' do
+        backend.should_receive(:invalid_redis_command).and_raise Redis::CommandError
+        expect { wrapper.invalid_redis_command }.to raise_error(Redis::CommandError)
+      end
+
+      it 'raises a Raidis::ConnectionError if the backend could not be instantiated' do
+        wrapper.stub!(:redis)
+        expect { wrapper.redis_command }.to raise_error(Raidis::ConnectionError)
+      end
+
+      it 'wraps the call in reloading_connection' do
+        wrapper.should_receive(:reloading_connection).with(no_args()).and_return 'some_value'
+        wrapper.get(:some_key).should == 'some_value'
+      end
+
+      it 'wraps the call in observing_connection' do
+        wrapper.should_receive(:observing_connection).with(no_args()).and_return 'some_value'
+        wrapper.get(:some_key).should == 'some_value'
+      end
+    end
+
+    context 'with an unstable connection' do
+      before do
+        wrapper.stub!(:redis!).and_return shaky_backend
+      end
+
+      it 'retrieves the result even when the backend fails several times' do
+        wrapper.some_redis_command.should == :shaky_result
+      end
+
+      context 'when running out of retries' do
+        before do
+          config.stub!(:retries).and_return 2
+        end
+
+        it 'finally raises a unified connection error' do
+          expect { wrapper.some_redis_command }.to raise_error(Raidis::ConnectionError)
+        end
+      end
+    end
+  end
+
+end
+
+

data/spec/spec_helper.rb

@@ -0,0 +1,47 @@
+require 'timecop'
+require 'raidis'
+
+def ensure_class_or_module(full_name, class_or_module)
+  full_name.to_s.split(/::/).inject(Object) do |context, name|
+    begin
+      context.const_get(name)
+    rescue NameError
+      if class_or_module == :class
+        context.const_set(name, Class.new)
+      else
+        context.const_set(name, Module.new)
+      end
+    end
+  end
+end
+
+def ensure_module(name)
+  ensure_class_or_module(name, :module)
+end
+
+def ensure_class(name)
+  ensure_class_or_module(name, :class)
+end
+
+ensure_module :Trouble
+
+RSpec.configure do |config|
+
+  # Global before hook
+  config.before do
+    Trouble.stub!(:notify)
+
+    Raidis.configure do |config|
+      config.redis_db = 15
+      config.redis_timeout = 0.5
+    end
+    Raidis.config.stub!(:info_file_path).and_return mock(:info_file_path, exist?: true, readable?: true, read: '127.0.0.1')
+  end
+
+  # Global after hooks
+  config.after do
+    Raidis.reset!
+    Timecop.return
+  end
+
+end

metadata

@@ -0,0 +1,132 @@
+--- !ruby/object:Gem::Specification
+name: raidis
+version: !ruby/object:Gem::Version
+  version: 0.1.0
+  prerelease: 
+platform: ruby
+authors:
+- bukowskis
+autorequire: 
+bindir: bin
+cert_chain: []
+date: 2013-03-26 00:00:00.000000000 Z
+dependencies:
+- !ruby/object:Gem::Dependency
+  name: redis-namespace
+  requirement: !ruby/object:Gem::Requirement
+    none: false
+    requirements:
+    - - ! '>='
+      - !ruby/object:Gem::Version
+        version: '0'
+  type: :runtime
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    none: false
+    requirements:
+    - - ! '>='
+      - !ruby/object:Gem::Version
+        version: '0'
+- !ruby/object:Gem::Dependency
+  name: rspec
+  requirement: !ruby/object:Gem::Requirement
+    none: false
+    requirements:
+    - - ! '>='
+      - !ruby/object:Gem::Version
+        version: '0'
+  type: :development
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    none: false
+    requirements:
+    - - ! '>='
+      - !ruby/object:Gem::Version
+        version: '0'
+- !ruby/object:Gem::Dependency
+  name: guard-rspec
+  requirement: !ruby/object:Gem::Requirement
+    none: false
+    requirements:
+    - - ! '>='
+      - !ruby/object:Gem::Version
+        version: '0'
+  type: :development
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    none: false
+    requirements:
+    - - ! '>='
+      - !ruby/object:Gem::Version
+        version: '0'
+- !ruby/object:Gem::Dependency
+  name: rb-fsevent
+  requirement: !ruby/object:Gem::Requirement
+    none: false
+    requirements:
+    - - ! '>='
+      - !ruby/object:Gem::Version
+        version: '0'
+  type: :development
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    none: false
+    requirements:
+    - - ! '>='
+      - !ruby/object:Gem::Version
+        version: '0'
+- !ruby/object:Gem::Dependency
+  name: timecop
+  requirement: !ruby/object:Gem::Requirement
+    none: false
+    requirements:
+    - - ! '>='
+      - !ruby/object:Gem::Version
+        version: '0'
+  type: :development
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    none: false
+    requirements:
+    - - ! '>='
+      - !ruby/object:Gem::Version
+        version: '0'
+description: See https://github.com/bukowskis/raidis
+email: 
+executables: []
+extensions: []
+extra_rdoc_files: []
+files:
+- lib/raidis/availability.rb
+- lib/raidis/configuration.rb
+- lib/raidis/redis_wrapper.rb
+- lib/raidis.rb
+- spec/lib/raidis_spec.rb
+- spec/lib/redis_wrapper_spec.rb
+- spec/spec_helper.rb
+- README.md
+homepage: https://github.com/bukowskis/raidis
+licenses: []
+post_install_message: 
+rdoc_options: []
+require_paths:
+- lib
+required_ruby_version: !ruby/object:Gem::Requirement
+  none: false
+  requirements:
+  - - ! '>='
+    - !ruby/object:Gem::Version
+      version: '0'
+required_rubygems_version: !ruby/object:Gem::Requirement
+  none: false
+  requirements:
+  - - ! '>='
+    - !ruby/object:Gem::Version
+      version: '0'
+requirements: []
+rubyforge_project: 
+rubygems_version: 1.8.23
+signing_key: 
+specification_version: 3
+summary: Yet another failover wrapper around Redis.
+test_files: []