semian 0.0.4 → 0.0.5

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA1:
-  metadata.gz: 04219b7eea123cad23d49905ec52c376484013fb
-  data.tar.gz: 4922f4c4f148425ebb7d1ced40dc8bf6fc3454ec
+  metadata.gz: 2ea7b3c7c6d5ba837117b012f0acd14d061a055f
+  data.tar.gz: bfd642ac2f32a39912d4d0696e1454ae218b2484
 SHA512:
-  metadata.gz: 896373f2c1401134888a8ac3d503663bcbec528f7d304173196310ec7a585f3e5c01faa0c92ce199ba935b8324e4fa57a1f177f77503bfcd59219e64f576952e
-  data.tar.gz: ea74197c538ba50e31c01a518570f3bb94f60d5f28be77554616dc9c2595cbf4be24e280cff612d66ef0c970737f025971f93316c2c6a9a3d9a83e1472698e20
+  metadata.gz: 3339e53e0c0b4170b27194366c8b1f1b8fb3f0a24483ed5b886b5ea9886024dfeb8c242391ca23f68959605d46c94e67d463741616afc25b05740abe32b4be7e
+  data.tar.gz: 27db27c22c66ab98908bf84efb1fcee3809f17280cd61a01eb2e547a5eaf1430c7124b598dc7a53c25664c9cf090ae9712ae8a884247673ee8f4e63c2f01a88b

data/Gemfile.lock

@@ -1,7 +1,7 @@
 PATH
   remote: .
   specs:
-    semian (0.0.4)
+    semian (0.0.5)
 
 GEM
   remote: https://rubygems.org/

data/README.md

@@ -1,31 +1,83 @@
-## Semian
+## Semian [![Build Status](https://travis-ci.org/Shopify/semian.svg?branch=master)](https://travis-ci.org/Shopify/semian)
 
-[![Build Status](https://travis-ci.org/csfrancis/semian.svg?branch=master)](https://travis-ci.org/csfrancis/semian)
+Semian is a Ruby implementation of the Bulkhead resource isolation pattern,
+using SysV semaphores. Bulkheading controls access to external resources,
+protecting against resource or network latency, by allowing otherwise slow
+queries to fail fast.
 
-Inspired by the bulkhead resource isolation pattern used in [Hystrix](https://github.com/Netflix/Hystrix/wiki/How-it-Works#Isolation), Semian aims to provide a Ruby API that can be used to control access to external resources.
+Downtime is easy to detect. Requests fail when querying the resource, usually
+fast. Reliably detecting higher than normal latency is more difficult. Strict
+timeouts is one solution, but picking those are hard and usually needs to be
+done per query or section of your application.
 
-This can be used with a forking Ruby application server like Unicorn to prevent app server starvation when a resource is slow or not responding.
+Semian takes a different approach. Instead of asking the question: "How long can
+my query execute?" it raises the question "How long do I want to wait before
+starting to execute my query?".
+
+Imagine that your database is very slow. Requests that hit the slow database are
+processed in your workers and end up timing out at the worker level. However,
+other requests don't touch the slow database. These requests will start to queue
+up behind the requests to the slow database, possibly never being served
+because the client disconnects due to slowness. You're now effectively down,
+because a single external resource is slow.
+
+Semian solves this problem with resource tickets. Every time a worker addresses
+an external resource, it takes a ticket for the duration of the query.  When the
+query returns, it puts the ticket back into the pool. If you have `n` tickets,
+and the `n + 1` worker tries to acquire a ticket to query the resource it'll
+wait for `timeout` seconds to see if a ticket comes available, otherwise it'll
+raise `Semian::TimeoutError`. 
+
+By failing fast, this solves the problem of one slow database taking your
+platform down. The busyness of the external resource determines the `timeout`
+and ticket count. You can also rescue `Semian::TimeoutError` to provide fallback
+values, such as showing an error message to the user.
+
+A subset of workers will still be tied up on the slow database, meaning you are
+under capacity with a slow external resource. However, at most you'll have
+`ticket count` workers occupied. This is a small price to pay. By implementing
+the circuit breaker pattern on top of Semian, you can avoid that. That may be
+built into Semian in the future.
+
+Under the hood, Semian is implemented with SysV semaphores. In a threaded web
+server, the semaphore could be in-process. Semian was written with forked web
+servers in mind, such as Unicorn, but Semian can be used perfectly fine in a
+threaded web server.
 
 ### Usage
 
-In a master process, register a resource with a specified number of tickets (number of concurrent clients):
-```ruby
-require 'semian'
+In a master process, register a resource with a specified number of tickets
+(number of concurrent clients):
 
+```ruby
 Semian.register(:mysql_master, tickets: 3, timeout: 0.5)
 ```
 
 Then in your child processes, you can use the resource:
+
 ```ruby
 Semian[:mysql_master].acquire do
-	# Query mysql and do things
+  # Query the database. If three other workers are querying this resource at the
+  # same time, this block will block for up to 0.5s waiting for another worker
+  # to release a ticket. Otherwise, it'll raise `Semian::TimeoutError`.
 end
 ```
 
-If you have a process that does not fork, you can still use the same namespace to control access to a shared resource:
+If you have a process that doesn't `fork`, you can still use the same namespace
+to control access to a shared resource:
+
 ```ruby
 Semian.register(:mysql_master, timeout: 0.5)
+
 Semian[:mysql_master].acquire do
-	# Query mysql and do things
+  # Query the resource
 end
 ```
+
+### Install
+
+In your Gemfile:
+
+```ruby
+gem "semian"
+```

data/Rakefile

@@ -14,12 +14,18 @@ end
 # Ruby Extension
 # ==========================================================
 
-require 'rake/extensiontask'
-Rake::ExtensionTask.new('semian', GEMSPEC) do |ext|
-  ext.ext_dir = 'ext/semian'
-  ext.lib_dir = 'lib/semian'
+$:.unshift File.expand_path("../lib", __FILE__)
+require 'semian/platform'
+if Semian.supported_platform?
+  require 'rake/extensiontask'
+  Rake::ExtensionTask.new('semian', GEMSPEC) do |ext|
+    ext.ext_dir = 'ext/semian'
+    ext.lib_dir = 'lib/semian'
+  end
+  task :build => :compile
+else
+  task :build do; end
 end
-task :build => :compile
 
 # ==========================================================
 # Testing
@@ -27,7 +33,11 @@ task :build => :compile
 
 require 'rake/testtask'
 Rake::TestTask.new 'test' do |t|
-  t.test_files = FileList['test/test_*.rb']
+  t.test_files = if Semian.supported_platform?
+    FileList['test/test_semian.rb']
+  else
+    FileList['test/test_unsupported.rb']
+  end
 end
 task :test => :build
 

data/ext/semian/semian.c

@@ -33,7 +33,7 @@ typedef VALUE (*my_blocking_fn_t)(void*);
 
 static ID id_timeout;
 static VALUE eSyscall, eTimeout, eInternal;
-static int system_max_sempahore_count;
+static int system_max_semaphore_count;
 
 static const int kIndexTickets = 0;
 static const int kIndexTicketMax = 1;
@@ -116,7 +116,7 @@ semian_resource_alloc(VALUE klass)
 }
 
 static void
-set_sempahore_permissions(int sem_id, int permissions)
+set_semaphore_permissions(int sem_id, int permissions)
 {
   union semun sem_opts;
   struct semid_ds stat_buf;
@@ -202,7 +202,7 @@ configure_tickets(int sem_id, int tickets, int should_initialize)
     }
   } else if (tickets > 0) {
     /* it's possible that we haven't actually initialized the
-       sempahore structure yet - wait a bit in that case */
+       semaphore structure yet - wait a bit in that case */
     if (get_max_tickets(sem_id) == 0) {
       gettimeofday(&start_time, NULL);
       while (get_max_tickets(sem_id) == 0) {
@@ -288,8 +288,8 @@ semian_resource_initialize(VALUE self, VALUE id, VALUE tickets, VALUE permission
   if (TYPE(default_timeout) != T_FIXNUM && TYPE(default_timeout) != T_FLOAT) {
     rb_raise(rb_eTypeError, "expected numeric type for default_timeout");
   }
-  if (FIX2LONG(tickets) < 0 || FIX2LONG(tickets) > system_max_sempahore_count) {
-    rb_raise(rb_eArgError, "ticket count must be a non-negative value and less than %d", system_max_sempahore_count);
+  if (FIX2LONG(tickets) < 0 || FIX2LONG(tickets) > system_max_semaphore_count) {
+    rb_raise(rb_eArgError, "ticket count must be a non-negative value and less than %d", system_max_semaphore_count);
   }
   if (NUM2DBL(default_timeout) < 0) {
     rb_raise(rb_eArgError, "default timeout must be non-negative value");
@@ -312,7 +312,7 @@ semian_resource_initialize(VALUE self, VALUE id, VALUE tickets, VALUE permission
 
   configure_tickets(res->sem_id, FIX2LONG(tickets), created);
 
-  set_sempahore_permissions(res->sem_id, FIX2LONG(permissions));
+  set_semaphore_permissions(res->sem_id, FIX2LONG(permissions));
 
   return self;
 }
@@ -329,7 +329,7 @@ cleanup_semian_resource_acquire(VALUE self)
 }
 
 static void *
-acquire_sempahore_without_gvl(void *p)
+acquire_semaphore_without_gvl(void *p)
 {
   semian_resource_t *res = (semian_resource_t *) p;
   res->error = 0;
@@ -377,7 +377,7 @@ semian_resource_acquire(int argc, VALUE *argv, VALUE self)
   }
 
   /* release the GVL to acquire the semaphore */
-  WITHOUT_GVL(acquire_sempahore_without_gvl, &res, RUBY_UBF_IO, NULL);
+  WITHOUT_GVL(acquire_semaphore_without_gvl, &res, RUBY_UBF_IO, NULL);
   if (res.error != 0) {
     if (res.error == EAGAIN) {
       rb_raise(eTimeout, "timed out waiting for resource '%s'", res.name);
@@ -504,10 +504,10 @@ void Init_semian()
   id_timeout = rb_intern("timeout");
 
   if (semctl(0, 0, SEM_INFO, &info_buf) == -1) {
-    rb_raise(eInternal, "unable to determine maximum sempahore count - semctl() returned %d: %s ", errno, strerror(errno));
+    rb_raise(eInternal, "unable to determine maximum semaphore count - semctl() returned %d: %s ", errno, strerror(errno));
   }
-  system_max_sempahore_count = info_buf.semvmx;
+  system_max_semaphore_count = info_buf.semvmx;
 
   /* Maximum number of tickets available on this system. */
-  rb_define_const(cSemian, "MAX_TICKETS", INT2FIX(system_max_sempahore_count));
+  rb_define_const(cSemian, "MAX_TICKETS", INT2FIX(system_max_semaphore_count));
 }

data/lib/semian.rb

@@ -1,5 +1,3 @@
-require 'semian/semian'
-
 #
 # === Overview
 #
@@ -68,7 +66,7 @@ class Semian
     # +timeout+: Default timeout in seconds.
     #
     # Returns the registered resource.
-    def register(name, tickets: 0, permissions: 0600, timeout: 1)
+    def register(name, tickets: 0, permissions: 0660, timeout: 1)
       resource = Resource.new(name, tickets, permissions, timeout)
       resources[name] = resource
     end
@@ -85,4 +83,11 @@ class Semian
   end
 end
 
+require 'semian/platform'
+if Semian.supported_platform?
+  require 'semian/semian'
+else
+  require 'semian/unsupported'
+  $stderr.puts "Semian is not supported on #{RUBY_PLATFORM} - all operations will no-op"
+end
 require 'semian/version'

data/lib/semian/platform.rb

@@ -0,0 +1,6 @@
+class Semian
+  # Determines if Semian supported on the current platform.
+  def self.supported_platform?
+    RUBY_PLATFORM.end_with?('-linux')
+  end
+end

data/lib/semian/unsupported.rb

@@ -0,0 +1,21 @@
+class Semian
+  MAX_TICKETS = 0
+
+  class Resource #:nodoc:
+    def initialize(name, tickets, permissions, timeout); end
+
+    def destroy; end
+
+    def acquire
+      yield self
+    end
+
+    def count
+      0
+    end
+
+    def semid
+      0
+    end
+  end
+end

data/lib/semian/version.rb

@@ -1,3 +1,3 @@
 class Semian
-  VERSION = '0.0.4'
+  VERSION = '0.0.5'
 end

data/semian.gemspec

@@ -5,13 +5,13 @@ require 'semian/version'
 Gem::Specification.new do |s|
   s.name = 'semian'
   s.version = Semian::VERSION
-  s.summary = 'SysV semaphore based library for shared resource control'
+  s.summary = 'Bulkheading for Ruby with SysV semaphores'
   s.description = <<-DOC
     A Ruby C extention that is used to control access to shared resources
-    across process boundaries.
+    across process boundaries with SysV semaphores.
   DOC
   s.homepage = 'https://github.com/csfrancis/semian'
-  s.authors = 'Scott Francis'
+  s.authors = ['Scott Francis', 'Simon Eskildsen']
   s.email   = 'scott.francis@shopify.com'
   s.license = 'MIT'
 

data/test/test_unsupported.rb

@@ -0,0 +1,11 @@
+require 'test/unit'
+require 'semian'
+
+class TestSemian < Test::Unit::TestCase
+  def test_unsupported_acquire_yields
+    acquired = false
+    Semian.register :testing, tickets: 1
+    Semian[:testing].acquire { acquired = true }
+    assert acquired
+  end
+end

metadata

@@ -1,14 +1,15 @@
 --- !ruby/object:Gem::Specification
 name: semian
 version: !ruby/object:Gem::Version
-  version: 0.0.4
+  version: 0.0.5
 platform: ruby
 authors:
 - Scott Francis
+- Simon Eskildsen
 autorequire: 
 bindir: bin
 cert_chain: []
-date: 2014-10-05 00:00:00.000000000 Z
+date: 2014-11-03 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: rake-compiler
@@ -26,7 +27,7 @@ dependencies:
         version: '0.9'
 description: |2
       A Ruby C extention that is used to control access to shared resources
-      across process boundaries.
+      across process boundaries with SysV semaphores.
 email: scott.francis@shopify.com
 executables: []
 extensions:
@@ -43,9 +44,12 @@ files:
 - ext/semian/extconf.rb
 - ext/semian/semian.c
 - lib/semian.rb
+- lib/semian/platform.rb
+- lib/semian/unsupported.rb
 - lib/semian/version.rb
 - semian.gemspec
 - test/test_semian.rb
+- test/test_unsupported.rb
 homepage: https://github.com/csfrancis/semian
 licenses:
 - MIT
@@ -69,5 +73,5 @@ rubyforge_project:
 rubygems_version: 2.2.2
 signing_key: 
 specification_version: 4
-summary: SysV semaphore based library for shared resource control
+summary: Bulkheading for Ruby with SysV semaphores
 test_files: []