semian 0.2.0 → 0.3.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA1:
-  metadata.gz: 1da1ae8c9c9376d81347eae49e8755aee743acf1
-  data.tar.gz: dc3b57ca4747df54e05b072c3df52c7e731088a9
+  metadata.gz: fc4fd1a356e755e2aa60896da8d30394e545d44a
+  data.tar.gz: ea85970b77be890bf35c69a9d047525280f60e31
 SHA512:
-  metadata.gz: 880796b7330efc3566fbe66a1d1756f99617cc496640ba58d4277e83f401ea65ed5619ed60869ef67049c91d87f620b28f04fdd0015ceb09e5ac4306a33d2a73
-  data.tar.gz: 179a799c1b5d3e07abeb550ee940c527da01781ceceb6c1fd04dd2d42ca5971bde46a886449d4a1739da18944a25323aba593a064526f9ad464a51cdc4f19527
+  metadata.gz: 455a0fb4fef078ea9a7c2a5c5ee63584492726aa32e98d2a062a274d7680d6abb2366120722ffb20aa3069bc2efb282ac0906dc20af5dd656df8a588e5cfde63
+  data.tar.gz: bfe6ab61e67e6c5f6426bd0bf18b5710c50711f82966136bb8da420a2da2c091191886f57b834f130babff966620e3f717c16d6f65ed3e9d4dbd67f09c93ea74

data/.gitignore

@@ -5,3 +5,4 @@
 *.gem
 /html/
 Gemfile.lock
+vendor/

data/README.md

@@ -1,83 +1,121 @@
 ## Semian [![Build Status](https://travis-ci.org/Shopify/semian.svg?branch=master)](https://travis-ci.org/Shopify/semian)
 
-Semian is a Ruby implementation of the Bulkhead resource isolation pattern,
-using SysV semaphores. Bulkheading controls access to external resources,
-protecting against resource or network latency, by allowing otherwise slow
-queries to fail fast.
-
-Downtime is easy to detect. Requests fail when querying the resource, usually
-fast. Reliably detecting higher than normal latency is more difficult. Strict
-timeouts is one solution, but picking those are hard and usually needs to be
-done per query or section of your application.
-
-Semian takes a different approach. Instead of asking the question: "How long can
-my query execute?" it raises the question "How long do I want to wait before
-starting to execute my query?".
-
-Imagine that your database is very slow. Requests that hit the slow database are
-processed in your workers and end up timing out at the worker level. However,
-other requests don't touch the slow database. These requests will start to queue
-up behind the requests to the slow database, possibly never being served
-because the client disconnects due to slowness. You're now effectively down,
-because a single external resource is slow.
-
-Semian solves this problem with resource tickets. Every time a worker addresses
-an external resource, it takes a ticket for the duration of the query.  When the
-query returns, it puts the ticket back into the pool. If you have `n` tickets,
-and the `n + 1` worker tries to acquire a ticket to query the resource it'll
-wait for `timeout` seconds to see if a ticket comes available, otherwise it'll
-raise `Semian::TimeoutError`. 
-
-By failing fast, this solves the problem of one slow database taking your
-platform down. The busyness of the external resource determines the `timeout`
-and ticket count. You can also rescue `Semian::TimeoutError` to provide fallback
-values, such as showing an error message to the user.
-
-A subset of workers will still be tied up on the slow database, meaning you are
-under capacity with a slow external resource. However, at most you'll have
-`ticket count` workers occupied. This is a small price to pay. By implementing
-the circuit breaker pattern on top of Semian, you can avoid that. That may be
-built into Semian in the future.
-
-Under the hood, Semian is implemented with SysV semaphores. In a threaded web
-server, the semaphore could be in-process. Semian was written with forked web
-servers in mind, such as Unicorn, but Semian can be used perfectly fine in a
-threaded web server.
-
-### Usage
-
-In a master process, register a resource with a specified number of tickets
-(number of concurrent clients):
+Semian is a latency and fault tolerance library for protecting your Ruby
+applications against misbehaving external services. It allows you to fail fast
+so you can handle errors gracefully. The patterns are inspired by
+[Hystrix][hystrix] and [Release It][release-it]. Semian is an extraction from
+[Shopify][shopify] where it's been running successfully in production since
+October, 2014.
+
+For an overview of building resilient Ruby application, see [the blog post on
+Toxiproxy and Semian][resiliency-blog-post]. We recommend using
+[Toxiproxy][toxiproxy] to test for resiliency.
+
+# Usage
+
+Install by adding the gem to your `Gemfile` and require the [adapters](#adapters) you need:
 
 ```ruby
-Semian.register(:mysql_master tickets: 3, timeout: 0.5, error_threshold: 3, error_timeout: 10, success_threshold: 2)
+gem 'semian', require: %w(semian semian/mysql2 semian/redis)
 ```
 
-Then in your child processes, you can use the resource:
+We recommend this pattern of requiring adapters directly from the `Gemfile`.
+This makes ensures Semian adapters is loaded as early as possible, to also
+protect your application during boot. Please see the [adapter configuration
+section](#configuration) on how to configure adapters.
+
+## Adapters
+
+The following adapters are in Semian and work against the public gems:
+
+* [`semian/mysql2`][mysql-semian-adapter] (~> 0.3.16)
+* [`semian/redis`][redis-semian-adapter] (~> 3.2.1)
+
+### Configuration
+
+When instantiating a resource it now needs to be configured for Semian. This is
+done by passing `semian` as an argument when initializing the client. Examples
+built in adapters:
 
 ```ruby
-Semian[:mysql_master].acquire do
-  # Query the database. If three other workers are querying this resource at the
-  # same time, this block will block for up to 0.5s waiting for another worker
-  # to release a ticket. Otherwise, it'll raise `Semian::TimeoutError`.
-end
+# MySQL2 client
+# In Rails this means having a Semian key in database.yml for each db.
+client = Mysql2::Client.new(host: "localhost", username: "root", semian: {
+  name: "master",
+  tickets: 8, # See the Understanding Semian section on picking these values
+  success_threshold: 2,
+  error_threshold: 3,
+  error_timeout: 10
+})
+
+# Redis client
+client = Redis.new(semian: {
+  name: "inventory",
+  tickets: 4,
+  success_threshold: 2,
+  error_threshold: 4,
+  error_timeout: 20
+})
 ```
 
-If you have a process that doesn't `fork`, you can still use the same namespace
-to control access to a shared resource:
+### Creating an adapter
+
+To create a Semian adapter you must implement the following methods:
+
+1. [`include Semian::Adapter`][semian-adapter]. Use the helpers to wrap the
+   resource. This takes care of situations such as monitoring, nested resources,
+   unsupported platforms, creating the Semian resource if it doesn't already
+   exist and so on.
+2. `#semian_identifier`. This is responsible for returning a symbol that
+   represents every unique resource, for example `redis_master` or
+   `mysql_shard_1`. This is usually assembled from a `name` attribute on the
+   Semian configuration hash, but could also be `<host>:<port>`.
+3. `connect`. The name of this method varies. You must override the driver's
+   connect method with one that wraps the connect call with
+   `Semian::Resource#acquire`. You should do this at the lowest possible level.
+4. `query`. Same as `connect` but for queries on the resource.
+5. Define exceptions `ResourceBusyError` and `CircuitOpenError`. These are
+   raised when the request was rejected early because the resource is out of
+   tickets or because the circuit breaker is open (see [Understanding
+   Semian](#understanding-semian). They should inherit from the base exception
+   class from the raw driver. For example `Mysql2::Error` or
+   `Redis::BaseConnectionError` for the MySQL and Redis drivers. This makes it
+   easy to `rescue` and handle them gracefully in application code, by
+   `rescue`ing the base class.
+
+The best resource is looking at the [already implemented adapters](#adapters).
+
+## Monitoring
+
+With [`Semian::Instrumentable`][semian-instrumentable] clients can monitor
+Semian internals. For example to instrument just events with
+[`statsd-instrument`][statsd-instrument]:
 
 ```ruby
-Semian.register(:mysql_master, timeout: 0.5)
-
-Semian[:mysql_master].acquire do
-  # Query the resource
+# `event` is `success`, `busy`, `circuit_open`.
+# `resource` is the `Semian::Resource` object
+# `scope` is `connection` or `query` (others can be instrumented too from the adapter)
+# `adapter` is the name of the adapter (mysql2, redis, ..)
+Semian.subscribe do |event, resource, scope, adapter|
+  StatsD.increment("Shopify.#{adapter}.semian.#{event}", 1, tags: [
+    "resource:#{resource.name}",
+    "total_tickets:#{resource.tickets}",
+    "type:#{scope}",
+  ])
 end
 ```
 
-### Install
+# Understanding Semian
 
-In your Gemfile:
+Coming soon!
 
-```ruby
-gem "semian"
-```
+[hystrix]: https://github.com/Netflix/Hystrix
+[release-it]: https://pragprog.com/book/mnee/release-it
+[shopify]: http://www.shopify.com/
+[mysql-semian-adapter]: https://github.com/Shopify/semian/blob/master/lib/semian/mysql2.rb
+[redis-semian-adapter]: https://github.com/Shopify/semian/blob/master/lib/semian/redis.rb
+[semian-adapter]: https://github.com/Shopify/semian/blob/master/lib/semian/adapter.rb
+[semian-instrumentable]: https://github.com/Shopify/semian/blob/master/lib/semian/instrumentable.rb
+[statsd-instrument]: http://github.com/shopify/statsd-instrument
+[resiliency-blog-post]: http://www.shopify.com/technology/16906928-building-and-testing-resilient-ruby-on-rails-applications
+[toxiproxy]: https://github.com/Shopify/toxiproxy

data/Rakefile

@@ -18,7 +18,7 @@ end
 
 $:.unshift File.expand_path("../lib", __FILE__)
 require 'semian/platform'
-if Semian.supported_platform?
+if Semian.sysv_semaphores_supported?
   require 'rake/extensiontask'
   Rake::ExtensionTask.new('semian', GEMSPEC) do |ext|
     ext.ext_dir = 'ext/semian'

data/ext/semian/extconf.rb

@@ -2,7 +2,7 @@ $:.unshift File.expand_path("../../../lib", __FILE__)
 
 require 'semian/platform'
 
-unless Semian.supported_platform?
+unless Semian.sysv_semaphores_supported?
   File.write "Makefile", <<MAKEFILE
 all:
 clean:

data/lib/semian.rb

@@ -77,14 +77,19 @@ module Semian
   InternalError = Class.new(BaseError)
   OpenCircuitError = Class.new(BaseError)
 
+  def semaphores_enabled?
+    !ENV['SEMIAN_SEMAPHORES_DISABLED']
+  end
+
   module AdapterError
-    def initialize(semian_identifier, *args)
-      super(*args)
-      @semian_identifier = semian_identifier
-    end
+    attr_accessor :semian_identifier
 
     def to_s
-      "[#{@semian_identifier}] #{super}"
+      if @semian_identifier
+        "[#{@semian_identifier}] #{super}"
+      else
+        super
+      end
     end
   end
 
@@ -149,10 +154,11 @@ require 'semian/circuit_breaker'
 require 'semian/protected_resource'
 require 'semian/unprotected_resource'
 require 'semian/platform'
-if Semian.supported_platform?
+if Semian.sysv_semaphores_supported? && Semian.semaphores_enabled?
   require 'semian/semian'
 else
   Semian::MAX_TICKETS = 0
-  Semian.logger.info("Semian is not supported on #{RUBY_PLATFORM} - all operations will no-op")
+  Semian.logger.info("Semian sysv semaphores are not supported on #{RUBY_PLATFORM} - all operations will no-op") unless Semian.sysv_semaphores_supported?
+  Semian.logger.info("Semian semaphores are disabled, is this what you really want? - all operations will no-op") unless Semian.semaphores_enabled?
 end
 require 'semian/version'

data/lib/semian/adapter.rb

@@ -14,6 +14,8 @@ module Semian
       else
         options = semian_options.dup
         options.delete(:name)
+        options[:exceptions] ||= []
+        options[:exceptions] += resource_exceptions
         ::Semian.retrieve_or_register(semian_identifier, **options)
       end
     end
@@ -28,7 +30,10 @@ module Semian
     rescue ::Semian::OpenCircuitError => error
       raise self.class::CircuitOpenError.new(semian_identifier, error)
     rescue ::Semian::BaseError => error
-      raise self.class::ResourceOccupiedError.new(semian_identifier, error)
+      raise self.class::ResourceBusyError.new(semian_identifier, error)
+    rescue *resource_exceptions => error
+      error.semian_identifier = semian_identifier if error.respond_to?(:semian_identifier=)
+      raise
     end
 
     def semian_options
@@ -41,6 +46,10 @@ module Semian
       raise NotImplementedError.new("Semian adapters must implement a `raw_semian_options` method")
     end
 
+    def resource_exceptions
+      []
+    end
+
     def resource_already_acquired?
       @resource_acquired
     end

data/lib/semian/circuit_breaker.rb

@@ -25,14 +25,6 @@ module Semian
       result
     end
 
-    def with_fallback(fallback, &block)
-      acquire(&block)
-    rescue *@exceptions
-      evaluate_fallback(fallback)
-    rescue OpenCircuitError
-      evaluate_fallback(fallback)
-    end
-
     def request_allowed?
       return true if closed?
       half_open if error_timeout_expired?
@@ -63,14 +55,6 @@ module Semian
 
     private
 
-    def evaluate_fallback(fallback_value_or_block)
-      if fallback_value_or_block.respond_to?(:call)
-        fallback_value_or_block.call
-      else
-        fallback_value_or_block
-      end
-    end
-
     def closed?
       state == :closed
     end

data/lib/semian/mysql2.rb

@@ -3,11 +3,16 @@ require 'semian/adapter'
 require 'mysql2'
 
 module Mysql2
+  Mysql2::Error.include(::Semian::AdapterError)
+
   class SemianError < Mysql2::Error
-    include ::Semian::AdapterError
+    def initialize(semian_identifier, *args)
+      super(*args)
+      @semian_identifier = semian_identifier
+    end
   end
 
-  ResourceOccupiedError = Class.new(SemianError)
+  ResourceBusyError = Class.new(SemianError)
   CircuitOpenError = Class.new(SemianError)
 end
 
@@ -15,7 +20,13 @@ module Semian
   module Mysql2
     include Semian::Adapter
 
-    ResourceOccupiedError = ::Mysql2::ResourceOccupiedError
+    CONNECTION_ERROR = Regexp.union(
+      /Can't connect to MySQL server on/i,
+      /Lost connection to MySQL server during query/i,
+      /MySQL server has gone away/i,
+    )
+
+    ResourceBusyError = ::Mysql2::ResourceBusyError
     CircuitOpenError = ::Mysql2::CircuitOpenError
 
     DEFAULT_HOST = 'localhost'
@@ -51,6 +62,16 @@ module Semian
       acquire_semian_resource(adapter: :mysql, scope: :connection) { raw_connect(*args) }
     end
 
+    def acquire_semian_resource(*)
+      super
+    rescue ::Mysql2::Error => error
+      if error.message =~ CONNECTION_ERROR
+        semian_resource.mark_failed(error)
+        error.semian_identifier = semian_identifier
+      end
+      raise
+    end
+
     def raw_semian_options
       return query_options[:semian] if query_options.key?(:semian)
       return query_options['semian'.freeze] if query_options.key?('semian'.freeze)

data/lib/semian/platform.rb

@@ -1,6 +1,6 @@
 module Semian
   # Determines if Semian supported on the current platform.
-  def self.supported_platform?
+  def self.sysv_semaphores_supported?
     /linux/.match(RUBY_PLATFORM)
   end
 end

data/lib/semian/protected_resource.rb

@@ -5,7 +5,7 @@ module Semian
     extend Forwardable
 
     def_delegators :@resource, :destroy, :count, :semid, :tickets, :name
-    def_delegators :@circuit_breaker, :reset
+    def_delegators :@circuit_breaker, :reset, :mark_failed, :request_allowed?
 
     def initialize(resource, circuit_breaker)
       @resource = resource
@@ -20,7 +20,7 @@ module Semian
             yield self
           end
         rescue ::Semian::TimeoutError
-          Semian.notify(:occupied, self, scope, adapter)
+          Semian.notify(:busy, self, scope, adapter)
           raise
         end
       end
@@ -28,9 +28,5 @@ module Semian
       Semian.notify(:circuit_open, self, scope, adapter)
       raise
     end
-
-    def with_fallback(fallback, &block)
-      @circuit_breaker.with_fallback(fallback) { @resource.acquire(&block) }
-    end
   end
 end

data/lib/semian/redis.rb

@@ -3,19 +3,36 @@ require 'semian/adapter'
 require 'redis'
 
 class Redis
+  Redis::BaseConnectionError.include(::Semian::AdapterError)
+  ::Errno::EINVAL.include(::Semian::AdapterError)
+
   class SemianError < Redis::BaseConnectionError
-    include ::Semian::AdapterError
+    def initialize(semian_identifier, *args)
+      super(*args)
+      @semian_identifier = semian_identifier
+    end
   end
 
-  ResourceOccupiedError = Class.new(SemianError)
+  ResourceBusyError = Class.new(SemianError)
   CircuitOpenError = Class.new(SemianError)
+
+  # This memoized alias is necessary because during a `pipelined` block
+  # the client is replaced by an instance of `Redis::Pipeline` and there is
+  # no way to access the original client.
+  def semian_resource
+    @semian_resource ||= @client.semian_resource
+  end
+
+  def semian_identifier
+    semian_resource.name
+  end
 end
 
 module Semian
   module Redis
     include Semian::Adapter
 
-    ResourceOccupiedError = ::Redis::ResourceOccupiedError
+    ResourceBusyError = ::Redis::ResourceBusyError
     CircuitOpenError = ::Redis::CircuitOpenError
 
     # The naked methods are exposed as `raw_query` and `raw_connect` for instrumentation purpose
@@ -45,6 +62,13 @@ module Semian
 
     private
 
+    def resource_exceptions
+      [
+        ::Redis::BaseConnectionError,
+        ::Errno::EINVAL, # Hiredis bug: https://github.com/redis/hiredis-rb/issues/21
+      ]
+    end
+
     def raw_semian_options
       return options[:semian] if options.key?(:semian)
       return options['semian'.freeze] if options.key?('semian'.freeze)

data/lib/semian/unprotected_resource.rb

@@ -29,5 +29,12 @@ module Semian
 
     def reset
     end
+
+    def request_allowed?
+      true
+    end
+
+    def mark_failed(error)
+    end
   end
 end

data/lib/semian/version.rb

@@ -1,3 +1,3 @@
 module Semian
-  VERSION = '0.2.0'
+  VERSION = '0.3.0'
 end

data/test/circuit_breaker_test.rb

@@ -9,35 +9,6 @@ class TestCircuitBreaker < MiniTest::Unit::TestCase
     @resource = Semian[:testing]
   end
 
-  def test_with_fallback_value_returns_the_value
-    result = @resource.with_fallback(42) do
-      raise SomeError
-    end
-    assert_equal 42, result
-  end
-
-  def test_with_fallback_block_call_the_block
-    result = @resource.with_fallback(-> { 42 }) do
-      raise SomeError
-    end
-    assert_equal 42, result
-  end
-
-  def test_unknown_exceptions_are_not_rescued
-    assert_raises RuntimeError do
-      @resource.with_fallback(42) do
-        raise RuntimeError
-      end
-    end
-  end
-
-  def test_all_semian_exceptions_are_rescued
-    result = @resource.with_fallback(42) do
-      raise Semian::BaseError
-    end
-    assert_equal 42, result
-  end
-
   def test_acquire_yield_when_the_circuit_is_closed
     block_called = false
     @resource.acquire { block_called = true }
@@ -124,18 +95,23 @@ class TestCircuitBreaker < MiniTest::Unit::TestCase
   end
 
   def trigger_error!(resource = @resource)
-    resource.with_fallback(42) { raise SomeError }
+    resource.acquire { raise SomeError }
+  rescue SomeError
   end
 
   def assert_circuit_closed(resource = @resource)
     block_called = false
-    resource.with_fallback(42) { block_called = true }
+    resource.acquire { block_called = true }
     assert block_called, 'Expected the circuit to be closed, but it was open'
   end
 
   def assert_circuit_opened(resource = @resource)
-    block_called = false
-    resource.with_fallback(42) { block_called = true }
-    refute block_called, 'Expected the circuit to be open, but it was closed'
+    open = false
+    begin
+      resource.acquire { }
+    rescue Semian::OpenCircuitError
+      open = true
+    end
+    assert open, 'Expected the circuit to be open, but it was closed'
   end
 end

data/test/instrumentation_test.rb

@@ -6,8 +6,8 @@ class TestInstrumentation < MiniTest::Unit::TestCase
     Semian.register(:testing, tickets: 1, error_threshold: 1, error_timeout: 5, success_threshold: 1)
   end
 
-  def test_occupied_instrumentation
-    assert_notify(:success, :occupied) do
+  def test_busy_instrumentation
+    assert_notify(:success, :busy) do
       Semian[:testing].acquire do
         assert_raises Semian::TimeoutError do
           Semian[:testing].acquire {}
@@ -17,7 +17,7 @@ class TestInstrumentation < MiniTest::Unit::TestCase
   end
 
   def test_circuit_open_instrumentation
-    assert_notify(:success, :occupied) do
+    assert_notify(:success, :busy) do
       Semian[:testing].acquire do
         assert_raises Semian::TimeoutError do
           Semian[:testing].acquire {}

data/test/mysql2_test.rb

@@ -29,6 +29,29 @@ class TestMysql2 < MiniTest::Unit::TestCase
     assert_instance_of Semian::UnprotectedResource, resource
   end
 
+  def test_connection_errors_open_the_circuit
+    @proxy.downstream(:latency, latency: 1200).apply do
+      ERROR_THRESHOLD.times do
+        assert_raises ::Mysql2::Error do
+          connect_to_mysql!
+        end
+      end
+
+      assert_raises ::Mysql2::CircuitOpenError do
+        connect_to_mysql!
+      end
+    end
+  end
+
+  def test_query_errors_does_not_open_the_circuit
+    client = connect_to_mysql!
+    (ERROR_THRESHOLD * 2).times do
+      assert_raises ::Mysql2::Error do
+        client.query('ERROR!')
+      end
+    end
+  end
+
   def test_connect_instrumentation
     notified = false
     subscriber = Semian.subscribe do |event, resource, scope, adapter|
@@ -50,17 +73,37 @@ class TestMysql2 < MiniTest::Unit::TestCase
     client = connect_to_mysql!
 
     Semian[:mysql_testing].acquire do
-      assert_raises Mysql2::ResourceOccupiedError do
+      error = assert_raises Mysql2::ResourceBusyError do
         connect_to_mysql!
       end
+      assert_equal :mysql_testing, error.semian_identifier
+    end
+  end
+
+  def test_network_errors_are_tagged_with_the_resource_identifier
+    client = connect_to_mysql!
+    @proxy.down do
+      error = assert_raises ::Mysql2::Error do
+        client.query('SELECT 1 + 1;')
+      end
+      assert_equal client.semian_identifier, error.semian_identifier
+    end
+  end
+
+  def test_other_mysql_errors_are_not_tagged_with_the_resource_identifier
+    client = connect_to_mysql!
+
+    error = assert_raises Mysql2::Error do
+      client.query('SYNTAX ERROR!')
     end
+    assert_nil error.semian_identifier
   end
 
   def test_resource_timeout_on_connect
     @proxy.downstream(:latency, latency: 500).apply do
       background { connect_to_mysql! }
 
-      assert_raises Mysql2::ResourceOccupiedError do
+      assert_raises Mysql2::ResourceBusyError do
         connect_to_mysql!
       end
     end
@@ -71,7 +114,7 @@ class TestMysql2 < MiniTest::Unit::TestCase
       background { connect_to_mysql! }
 
       ERROR_THRESHOLD.times do
-        assert_raises Mysql2::ResourceOccupiedError do
+        assert_raises Mysql2::ResourceBusyError do
           connect_to_mysql!
         end
       end
@@ -111,7 +154,7 @@ class TestMysql2 < MiniTest::Unit::TestCase
     client = connect_to_mysql!
 
     Semian[:mysql_testing].acquire do
-      assert_raises Mysql2::ResourceOccupiedError do
+      assert_raises Mysql2::ResourceBusyError do
         client.query('SELECT 1 + 1;')
       end
     end
@@ -124,7 +167,7 @@ class TestMysql2 < MiniTest::Unit::TestCase
     @proxy.downstream(:latency, latency: 500).apply do
       background { client2.query('SELECT 1 + 1;') }
 
-      assert_raises Mysql2::ResourceOccupiedError do
+      assert_raises Mysql2::ResourceBusyError do
         client.query('SELECT 1 + 1;')
       end
     end
@@ -138,7 +181,7 @@ class TestMysql2 < MiniTest::Unit::TestCase
       background { client2.query('SELECT 1 + 1;') }
 
       ERROR_THRESHOLD.times do
-        assert_raises Mysql2::ResourceOccupiedError do
+        assert_raises Mysql2::ResourceBusyError do
           client.query('SELECT 1 + 1;')
         end
       end
@@ -163,7 +206,12 @@ class TestMysql2 < MiniTest::Unit::TestCase
   private
 
   def connect_to_mysql!(semian_options = {})
-    Mysql2::Client.new(host: '127.0.0.1', port: '13306', semian: SEMIAN_OPTIONS.merge(semian_options))
+    Mysql2::Client.new(
+      connect_timeout: 1,
+      host: '127.0.0.1',
+      port: '13306',
+      semian: SEMIAN_OPTIONS.merge(semian_options),
+    )
   end
 
   class FakeMysql < Mysql2::Client

data/test/redis_test.rb

@@ -24,11 +24,43 @@ class TestRedis < MiniTest::Unit::TestCase
     assert_equal :'redis_example.com:42/0', Redis.new(host: 'example.com', port: 42).client.semian_identifier
   end
 
+  def test_client_alias
+    redis = connect_to_redis!
+    assert_equal redis.client.semian_resource, redis.semian_resource
+    assert_equal redis.client.semian_identifier, redis.semian_identifier
+  end
+
   def test_semian_can_be_disabled
     resource = Redis.new(semian: false).client.semian_resource
     assert_instance_of Semian::UnprotectedResource, resource
   end
 
+  def test_connection_errors_open_the_circuit
+    client = connect_to_redis!
+
+    @proxy.downstream(:latency, latency: 600).apply do
+      ERROR_THRESHOLD.times do
+        assert_raises ::Redis::TimeoutError do
+          client.get('foo')
+        end
+      end
+
+      assert_raises ::Redis::CircuitOpenError do
+        client.get('foo')
+      end
+    end
+  end
+
+  def test_command_errors_does_not_open_the_circuit
+    client = connect_to_redis!
+    client.hset('my_hash', 'foo', 'bar')
+    (ERROR_THRESHOLD * 2).times do
+      assert_raises Redis::CommandError do
+        client.get('my_hash')
+      end
+    end
+  end
+
   def test_connect_instrumentation
     notified = false
     subscriber = Semian.subscribe do |event, resource, scope, adapter|
@@ -50,17 +82,36 @@ class TestRedis < MiniTest::Unit::TestCase
     client = connect_to_redis!
 
     Semian[:redis_testing].acquire do
-      assert_raises Redis::ResourceOccupiedError do
+      error = assert_raises Redis::ResourceBusyError do
+        connect_to_redis!
+      end
+      assert_equal :redis_testing, error.semian_identifier
+    end
+  end
+
+  def test_redis_connection_errors_are_tagged_with_the_resource_identifier
+    @proxy.downstream(:latency, latency: 600).apply do
+      error = assert_raises ::Redis::TimeoutError do
         connect_to_redis!
       end
+      assert_equal :redis_testing, error.semian_identifier
+    end
+  end
+
+  def test_other_redis_errors_are_not_tagged_with_the_resource_identifier
+    client = connect_to_redis!
+    client.set('foo', 'bar')
+    error = assert_raises ::Redis::CommandError do
+      client.hget('foo', 'bar')
     end
+    refute error.respond_to?(:semian_identifier)
   end
 
   def test_resource_timeout_on_connect
     @proxy.downstream(:latency, latency: 500).apply do
       background { connect_to_redis! }
 
-      assert_raises Redis::ResourceOccupiedError do
+      assert_raises Redis::ResourceBusyError do
         connect_to_redis!
       end
     end
@@ -71,7 +122,7 @@ class TestRedis < MiniTest::Unit::TestCase
       background { connect_to_redis! }
 
       ERROR_THRESHOLD.times do
-        assert_raises Redis::ResourceOccupiedError do
+        assert_raises Redis::ResourceBusyError do
           connect_to_redis!
         end
       end
@@ -111,7 +162,7 @@ class TestRedis < MiniTest::Unit::TestCase
     client = connect_to_redis!
 
     Semian[:redis_testing].acquire do
-      assert_raises Redis::ResourceOccupiedError do
+      assert_raises Redis::ResourceBusyError do
         client.get('foo')
       end
     end
@@ -124,7 +175,7 @@ class TestRedis < MiniTest::Unit::TestCase
     @proxy.downstream(:latency, latency: 500).apply do
       background { client2.get('foo') }
 
-      assert_raises Redis::ResourceOccupiedError do
+      assert_raises Redis::ResourceBusyError do
         client.get('foo')
       end
     end
@@ -140,7 +191,7 @@ class TestRedis < MiniTest::Unit::TestCase
       background { client2.get('foo') }
 
       ERROR_THRESHOLD.times do
-        assert_raises Redis::ResourceOccupiedError do
+        assert_raises Redis::ResourceBusyError do
           client.get('foo')
         end
       end
@@ -160,7 +211,14 @@ class TestRedis < MiniTest::Unit::TestCase
   private
 
   def connect_to_redis!(semian_options = {})
-    redis = Redis.new(host: '127.0.0.1', port: 16379, reconnect_attempts: 0, db: 1, semian: SEMIAN_OPTIONS.merge(semian_options))
+    redis = Redis.new(
+      host: '127.0.0.1',
+      port: 16379,
+      reconnect_attempts: 0,
+      db: 1,
+      timeout: 0.5,
+      semian: SEMIAN_OPTIONS.merge(semian_options)
+    )
     redis.client.connect
     redis
   end

data/test/{unsupported_test.rb → semian_test.rb}

@@ -19,4 +19,13 @@ class TestSemian < MiniTest::Unit::TestCase
     assert defined?(Semian::InternalError)
     assert defined?(Semian::Resource)
   end
+
+  def test_disabled_via_env_var
+    ENV['SEMIAN_SEMAPHORES_DISABLED'] = '1'
+
+    refute Semian.semaphores_enabled?
+  ensure
+    ENV.delete('SEMIAN_SEMAPHORES_DISABLED')
+  end
+
 end

data/test/unprotected_resource_test.rb

@@ -5,6 +5,11 @@ class UnprotectedResourceTest < MiniTest::Unit::TestCase
     @resource = Semian::UnprotectedResource.new(:foo)
   end
 
+  def test_interface_is_the_same
+    diff = Semian::ProtectedResource.public_instance_methods - Semian::UnprotectedResource.public_instance_methods
+    assert_equal [], diff
+  end
+
   def test_resource_name
     assert_equal :foo, @resource.name
   end
@@ -45,4 +50,11 @@ class UnprotectedResourceTest < MiniTest::Unit::TestCase
     assert acquired
   end
 
+  def test_request_is_allowed
+    assert @resource.request_allowed?
+  end
+
+  def test_mark_failed
+    @resource.mark_failed(:error)
+  end
 end

metadata

@@ -1,7 +1,7 @@
 --- !ruby/object:Gem::Specification
 name: semian
 version: !ruby/object:Gem::Version
-  version: 0.2.0
+  version: 0.3.0
 platform: ruby
 authors:
 - Scott Francis
@@ -31,7 +31,7 @@ cert_chain:
   fl3hbtVFTqbOlwL9vy1fudXcolIE/ZTcxQ+er07ZFZdKCXayR9PPs64heamfn0fp
   TConQSX2BnZdhIEYW+cKzEC/bLc=
   -----END CERTIFICATE-----
-date: 2015-02-19 00:00:00.000000000 Z
+date: 2015-03-11 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: rake-compiler
@@ -139,9 +139,9 @@ files:
 - test/mysql2_test.rb
 - test/redis_test.rb
 - test/resource_test.rb
+- test/semian_test.rb
 - test/test_helper.rb
 - test/unprotected_resource_test.rb
-- test/unsupported_test.rb
 homepage: https://github.com/csfrancis/semian
 licenses:
 - MIT