zoidberg 0.0.1 → 0.1.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA1:
-  metadata.gz: 1086df260c250afd0d140b888020efbff2f2a776
-  data.tar.gz: 350c9110094de3fb707ecfb12261c3899f9ca3d7
+  metadata.gz: f12f7686bfce42183b33d6a765027eed64113aee
+  data.tar.gz: 1b12f2ad1e6210a086c60c3dac02599cef678ba1
 SHA512:
-  metadata.gz: 9f53d61bb8dd1c6733322e9a29b6e2d049e517136b1b6f226ff1b129b2206c588e53d336cdac1fdc51c5299ce507a81e6d94f8923880dfae8df7b84cc695fe77
-  data.tar.gz: 9d2c5e02c1c2bae12bb5f37b4cc714f510880b002ed9397321dc8e230a09803f65dda44b917bf4ac66f7aaf5247aa6bbf5938bae0cc65228aa51940155cd0553
+  metadata.gz: 17f6a8e0cb57573076bbeb852c392699b8bdbde0f88b9818a213704cfb6e1c5c37e80cd6b207821e61f47f4d30e8825177e671136e209e2e197e1f0735063dd9
+  data.tar.gz: f8d4c6aba250fdf670e996bbb8668081cb21aa2ab0c8a049c7c8878f086af766e4525de47e2e1968c0b431985bdf3894ac05627a485b21fe78dbddb70dc048c9

data/CHANGELOG.md

@@ -0,0 +1,2 @@
+# v0.1.0
+* Initial release

data/CONTRIBUTING.md

@@ -0,0 +1,25 @@
+# Contributing
+
+## Branches
+
+### `master` branch
+
+The master branch is the current stable released version.
+
+### `develop` branch
+
+The develop branch is the current edge of development.
+
+## Pull requests
+
+* https://github.com/spox/zoidberg/pulls
+
+Please base all pull requests of the `develop` branch. Merges to
+`master` only occur through the `develop` branch. Pull requests
+based on `master` will likely be cherry picked.
+
+## Issues
+
+Need to report an issue? Use the github issues:
+
+* https://github.com/spox/zoidberg/issues

data/LICENSE

@@ -0,0 +1,13 @@
+   Copyright 2015 Chris Roberts
+
+   Licensed under the Apache License, Version 2.0 (the "License");
+   you may not use this file except in compliance with the License.
+   You may obtain a copy of the License at
+
+     http://www.apache.org/licenses/LICENSE-2.0
+
+   Unless required by applicable law or agreed to in writing, software
+   distributed under the License is distributed on an "AS IS" BASIS,
+   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+   See the License for the specific language governing permissions and
+   limitations under the License.

data/README.md

@@ -0,0 +1,151 @@
+# Zoidberg
+
+> Why not Zoidberg?
+
+## About
+
+Zoidberg is a small library attempting to provide synchronization
+and supervision without requiring any modifications to existing
+implementations. It is heavily inspired by Celluloid and while some
+APIs may look familiar they do not share a familiar implementation.
+
+## Usage
+
+Zoidberg provides a `Shell` which can be loaded into a class. After
+it has been loaded, new instances will provide implicit synchronization,
+which is nifty. For example, lets take a simple `Fubar` class that does
+a simple thing:
+
+```ruby
+class Fubar
+
+  attr_reader :string
+
+  def initialize
+    @string = ''
+    @chars = []
+  end
+
+  def append
+    string << char
+  end
+
+  private
+
+  def char
+    if(@chars.empty?)
+      @chars.replace (A..Z).to_a
+    end
+    @chars.shift
+  end
+
+end
+```
+
+Pretty simple class whose only purpose is to add characters to a string.
+And it does just that:
+
+```ruby
+inst = Fubar.new
+20.times{ inst.append }
+inst.string
+
+# => "ABCDEFGHIJKLMNOPQRST"
+```
+
+So this does exactly what we expect it to. Now, lets update this example and
+toss some threads into the mix:
+
+```ruby
+inst = Fubar.new
+20.times.map{ Thread.new{ inst.append } }.map(&:join)
+inst.string
+
+# => "ABCDEFGHIJKLMNOPQRST"
+```
+
+Cool, we get the same results! Looks like everything is great. Lets run it
+again to bask in this multi-threaded awesomeness!
+
+```ruby
+# => "AABCDEFGHIJKLMNOPQRS"
+```
+
+Hrm, that doesn't look quite right. It looks like there's an extra 'A' at the start. Maybe
+everything isn't so great. Lets try a few more:
+
+```ruby
+inst = Fubar.new
+100.times.map do
+  20.times.map{ Thread.new{ inst.append } }.map(&:join)
+end.uniq
+
+# => ["ABCDEFGHIJKLMNOPQRST", "ABCDEDGHIJKLMNOPQRST", "ACDEFGHIJKLMNOPQRST", "BCDEFGHIJKLMNOPQRST", "AABCDEFGHIJKLMNOPQRS", "ABCDEFHGIJKLMNOPQRST"]
+```
+
+Whelp, I don't even know what that is supposed to be, but it's certainly
+not what we are expecting. Well, we _are_ expecting it because this is
+an example on synchronization, but lets just pretend at this point we are
+amazed at this turn of events.
+
+To fix this, we need to add some synchronization so multiple threads aren't
+attempting to mutate state at the same time. But, instead of modifying the
+class and explicitly adding synchronization, lets see what happens when
+we toss `Zoidberg::Shell` into the mix (cause it's why everyone is here
+in the first place). We can just continue on with our previous examples
+and open up our defined class to inject the shell and re-run the example:
+
+```ruby
+require 'zoidberg'
+
+class Fubar
+  include Zoidberg::Shell
+end
+
+inst = Fubar.new
+20.times.map{ Thread.new{ inst.append } }.map(&:join)
+inst.string
+
+# => "ABCDEFGHIJKLMNOPQRST"
+```
+
+and running it lots of times we get:
+
+```ruby
+100.times.map{20.times.map{ Thread.new{ inst.append } }.map(&:join)}.uniq
+
+# => ["ABCDEFGHIJKLMNOPQRST"]
+```
+
+So this is pretty neat. We had a class that was shown to not be thread
+safe. We tossed a module into that class. Now that class is thread safe.
+
+## Features
+
+### Implicit Locking
+
+Zoidberg automatically synchronizes requests made to an instance. This
+behavior can be short circuited if the actual instance creates a thread
+and calls a method on itself. Otherwise, all external access to the
+instance will be automatically synchronized. Nifty.
+
+### Supervision
+
+Zoidberg provides lazy supervision. There is no single supervisor. Instead
+supervision is handled by the proxy which wraps the class. Failure of an
+instance will result in termination + reinstantiation. When externally
+accessing the instance nothing requires modification.
+
+### Pools
+
+Zoidberg allows pooling lazy supervised instances. Unexpected failures will
+cause the instance to be terminated and re-initialized as usual. The pool
+will deliver requests to free instances, or queue them until a free instance
+is available.
+
+### Garbage Collection
+
+Garbage collection happens as usual with Zoidberg. When an instance is created
+the result may look like the instance but really it is proxy wrapping the
+raw instance. When the proxy falls out of scope and is garbage collected the
+raw instance it wrapped will also fall out of scope and be garbage collected.

data/lib/zoidberg/future.rb

@@ -0,0 +1,34 @@
+require 'zoidberg'
+
+module Zoidberg
+  # Perform action and fetch result in the future
+  class Future
+
+    # @return [Thread] underlying thread running task
+    attr_reader :thread
+
+    # Create a new instance
+    #
+    # @yield block to execute
+    # @return [self]
+    def initialize(&block)
+      @thread = Thread.new(&block)
+    end
+
+    # @return [Object] result value
+    def value
+      unless(@value)
+        @value = @thread.value
+      end
+      @value
+    end
+
+    # Check if value is available
+    #
+    # @return [TrueClass, FalseClass]
+    def available?
+      !thread.alive?
+    end
+
+  end
+end

data/lib/zoidberg/logger.rb

@@ -0,0 +1,23 @@
+require 'zoidberg'
+require 'mono_logger'
+
+module Zoidberg
+  # Logger
+  class Logger < MonoLogger
+
+    # Quick override to ensure destination has append mode enabled if
+    # file io type
+    def initialize(logdev, *args)
+      if(logdev.respond_to?(:path))
+        begin
+          require 'fcntl'
+          unless(logdev.fcntl(Fcntl::F_GETFL) & Fcntl::O_APPEND == Fcntl::O_APPEND)
+            logdev = File.open(logdev.path, (File::WRONLY | File::APPEND))
+          end
+        rescue; end
+      end
+      super(logdev, *args)
+    end
+
+  end
+end

data/lib/zoidberg/pool.rb

@@ -0,0 +1,100 @@
+require 'zoidberg'
+
+module Zoidberg
+  # Populate a collection of instances and proxy requests to free
+  # instances within the pool
+  class Pool
+
+    include Zoidberg::Shell
+
+    # @return [Array<Object>] workers within pool
+    attr_reader :_workers
+    # @return [Signal] common signal for state updates
+    attr_reader :_signal
+
+    # Create a new pool instance. Provide class + instance
+    # initialization arguments when creating the pool. These will be
+    # used to build all instances within the pool.
+    #
+    # @return [self]
+    def initialize(*args, &block)
+      _validate_worker_class!(args.first)
+      @_signal = Signal.new
+      @_worker_count = 1
+      @_workers = []
+      @builder = lambda do
+        inst = args.first.new(
+          *args.slice(1, args.size),
+          &block
+        )
+        inst._zoidberg_signal = _signal
+        inst
+      end
+      _zoidberg_balance
+    end
+
+    # Validate worker class is properly supervised
+    #
+    # @raise [TypeError]
+    def _validate_worker_class!(klass)
+      unless(klass.ancestors.include?(Zoidberg::Supervise))
+        raise TypeError.new "Worker class `#{klass}` must include the `Zoidberg::Supervise` module!"
+      end
+    end
+
+    # Set or get the number of workers within the pool
+    #
+    # @param num [Integer]
+    # @return [Integer]
+    def _worker_count(num=nil)
+      if(num)
+        @_worker_count = num.to_i
+        _zoidberg_balance
+      end
+      @_worker_count
+    end
+
+    # Balance the pool to ensure the correct number of workers are
+    # available
+    #
+    # @return [TrueClass]
+    def _zoidberg_balance
+      unless(_workers.size == _worker_count)
+        if(_workers.size < _worker_count)
+          (_worker_count - _workers.size).times do
+            _workers << @builder.call
+          end
+        else
+          (_workers.size - _worker_count).times do
+            worker = _zoidberg_free_worker
+            worker._zoidberg_destroy!
+            _workers.delete(worker)
+          end
+        end
+      end
+      true
+    end
+
+    # Used to proxy request to worker
+    def method_missing(*args, &block)
+      worker = _zoidberg_free_worker
+      defer{ worker.send(*args, &block) }
+    end
+
+    # Find or wait for a free worker
+    #
+    # @return [Object]
+    def _zoidberg_free_worker
+      until(worker = @_workers.detect(&:_zoidberg_available?))
+        defer{ _signal.wait_for(:unlocked) }
+      end
+      worker
+    end
+
+    # Force termination of all workers when terminated
+    def terminate
+      @_workers.map(&:_zoidberg_destroy!)
+    end
+
+  end
+end

data/lib/zoidberg/proxy/confined.rb

@@ -0,0 +1,175 @@
+require 'fiber'
+require 'zoidberg'
+
+module Zoidberg
+  class Proxy
+    class Confined < Proxy
+
+      # @return [Thread] container thread
+      attr_reader :_source_thread
+      # @return [Queue] current request queue
+      attr_reader :_requests
+      # @return [TrueClass, FalseClass] blocked running task
+      attr_reader :_blocked
+
+      # Create a new isolation wrapper
+      #
+      # @param object [Object] object to wrap
+      # @return [self]
+      def initialize(klass, *args, &block)
+        @_requests = ::Queue.new
+        @_blocked = false
+        @_source_thread = ::Thread.new do
+          ::Zoidberg.logger.debug 'Starting the isolation request processor'
+          ::Thread.current[:root_fiber] = ::Fiber.current
+          _isolate!
+        end
+        @_build_args = [klass, *args, block]
+        @_raw_instance = klass.unshelled_new(*args, &block)
+        @_raw_instance._zoidberg_proxy(self)
+        if(@_raw_instance.class.include?(::Zoidberg::Supervise))
+          @_supervised = true
+        end
+        ::Zoidberg.logger.debug "Zoidberg object isolation wrap: #{@_build_args.inspect}"
+      end
+
+      # Call into instance asynchronously
+      #
+      # @note use caution with shared data using this method
+      def _async_request(blocking, method_name, *args, &block)
+        ::Zoidberg.logger.debug "Received async request from remote thread. Added to queue: #{_raw_instance.class}##{method_name}(#{args.map(&:inspect).join(', ')})"
+        _requests << ::Smash.new(
+          :uuid => ::Zoidberg.uuid,
+          :arguments => [method_name, *args],
+          :block => block,
+          :response => nil,
+          :async => true,
+          :blocking => blocking == :blocking
+        )
+        nil
+      end
+
+      # Wrapping for provided object
+      def method_missing(*args, &block)
+        res = nil
+        begin
+          if(::ENV['ZOIDBERG_TESTING'])
+            ::Kernel.require 'timeout'
+            ::Timeout.timeout(::ENV.fetch('ZOIDBERG_TESTING_TIMEOUT', 5).to_i) do
+              res = _isolated_request(*args, &block)
+            end
+          else
+            res = _isolated_request(*args, &block)
+          end
+        rescue ::Zoidberg::Supervise::AbortException => e
+          ::Kernel.raise e.original_exception
+        rescue ::Exception => e
+          _zoidberg_unexpected_error(e)
+          ::Zoidberg.logger.debug "Exception on: #{_raw_instance.class}##{args.first}(#{args.slice(1, args.size).map(&:inspect).join(', ')})"
+          ::Kernel.raise e
+        end
+        res
+      end
+
+      # Send the method request to the wrapped instance
+      #
+      # @param method_name [String, Symbol] method to call on instance
+      # @param args [Object] arguments for call
+      # @yield block for call
+      # @return [Object] result
+      def _isolated_request(method_name, *args, &block)
+        if(_source_thread == ::Thread.current)
+          ::Zoidberg.logger.debug "Received request from source thread: #{_raw_instance.class}##{method_name}(#{args.map(&:inspect).join(', ')})"
+          _raw_instance.__send__(method_name, *args, &block)
+        else
+          unless(_source_thread.alive?)
+            ::Kernel.raise ::Zoidberg::DeadException.new('Instance in terminated state!')
+          end
+          ::Zoidberg.logger.debug "Received request from remote thread. Added to queue: #{_raw_instance.class}##{method_name}(#{args.map(&:inspect).join(', ')})"
+          response_queue = ::Queue.new
+          _requests << ::Smash.new(
+            :uuid => ::Zoidberg.uuid,
+            :arguments => [method_name, *args],
+            :block => block,
+            :response => response_queue
+          )
+          result = response_queue.pop
+          if(result.is_a?(::Exception))
+            ::Kernel.raise result
+          else
+            result
+          end
+        end
+      end
+
+      def _zoidberg_available?
+        !_blocked
+      end
+
+      protected
+
+      # Process requests
+      def _isolate!
+        begin
+          ::Kernel.loop do
+            begin
+              _process_request(_requests.pop)
+            rescue => e
+              ::Zoidberg.logger.error "Unexpected looping error! (#{e.class}: #{e})"
+              ::Zoidberg.logger.error "#{e.class}: #{e}\n#{e.backtrace.join("\n")}"
+              ::Thread.main.raise e
+            end
+          end
+        ensure
+          until(_requests.empty)
+            requests.pop[:response] << ::Zoidberg::DeadException.new('Instance in terminated state!')
+          end
+        end
+      end
+
+      # Process a request
+      #
+      # @param request [Hash]
+      # @return [self]
+      def _process_request(request)
+        begin
+          @_blocked = !request[:async]
+          ::Zoidberg.logger.debug "Processing received request: #{request.inspect}"
+          unless(request[:task])
+            request[:task] = ::Zoidberg::Task.new(request[:async] ? :async : :serial, _raw_instance, [request]) do |req|
+              begin
+                result = origin.__send__(
+                  *req[:arguments],
+                  &req[:block]
+                )
+                if(req[:response])
+                  req[:response] << result
+                end
+              rescue ::Exception => exception
+                if(req[:response])
+                  req[:response] << exception
+                end
+              end
+            end
+          end
+          if(request[:task].waiting?)
+            if(_raw_instance.alive?)
+              request[:task].proceed
+              request[:task].value if request[:blocking]
+            else
+              request[:response] << ::Zoidberg::DeadException.new('Instance in terminated state!')
+              request[:task].halt!
+            end
+          end
+          _requests.push(request) unless request[:task].complete? || request[:async]
+          ::Zoidberg.logger.debug "Request processing completed. #{request.inspect}"
+        ensure
+          @_blocked = false
+        end
+        self
+      end
+
+    end
+
+  end
+end

data/lib/zoidberg/proxy/liberated.rb

@@ -0,0 +1,135 @@
+require 'zoidberg'
+
+module Zoidberg
+  class Proxy
+
+    class Liberated < Proxy
+
+      # @return [Thread] current owner of lock
+      attr_reader :_locker
+      # @return [Hash<Integer:Thread>]
+      attr_reader :_raw_threads
+
+      # Create a new proxy instance, new real instance, and link them
+      #
+      # @return [self]
+      def initialize(klass, *args, &block)
+        @_build_args = [klass, args, block]
+        @_lock = ::Mutex.new
+        @_count_lock = ::Mutex.new
+        @_accessing_threads = []
+        @_locker = nil
+        @_locker_count = 0
+        @_zoidberg_signal = nil
+        @_raw_instance = klass.unshelled_new(*args, &block)
+        @_raw_instance._zoidberg_proxy(self)
+        @_raw_threads = ::Smash.new{ ::Array.new }
+        if(@_raw_instance.class.ancestors.include?(::Zoidberg::Supervise))
+          @_supervised = true
+        end
+      end
+
+      # Used to proxy request to real instance
+      def method_missing(*args, &block)
+        res = nil
+        @_accessing_threads << ::Thread.current
+        begin
+          _aquire_lock!
+          if(::ENV['ZOIDBERG_TESTING'])
+            ::Kernel.require 'timeout'
+            ::Timeout.timeout(::ENV.fetch('ZOIDBERG_TESTING_TIMEOUT', 5).to_i) do
+              res = @_raw_instance.__send__(*args, &block)
+            end
+          else
+            res = @_raw_instance.__send__(*args, &block)
+          end
+        rescue ::Zoidberg::Supervise::AbortException => e
+          ::Kernel.raise e.original_exception
+        rescue ::Exception => e
+          ::Zoidberg.logger.debug "Exception on: #{_raw_instance.class.name}##{args.first}(#{args.slice(1, args.size).map(&:inspect).join(', ')})"
+          _zoidberg_unexpected_error(e)
+          if(e.class.to_s == 'fatal' && !@_fatal_retry)
+            @_fatal_retry = true
+            retry
+          else
+            ::Kernel.raise e
+          end
+        ensure
+          _release_lock!
+          t_idx = @_accessing_threads.index(::Thread.current)
+          @_accessing_threads.delete_at(t_idx) if t_idx
+        end
+        res
+      end
+
+      # @return [TrueClass, FalseClass] currently locked
+      def _zoidberg_locked?
+        @_lock && @_lock.locked?
+      end
+
+      # @return [TrueClass, FalseClass] currently unlocked
+      def _zoidberg_available?
+        !_zoidberg_locked?
+      end
+
+      # Aquire the lock to access real instance. If already locked, will
+      # wait until lock can be aquired.
+      #
+      # @return [TrueClas]
+      def _aquire_lock!
+        if(@_lock)
+          @_lock.lock unless @_locker == ::Thread.current
+          @_locker = ::Thread.current
+          @_locker_count += 1
+          _zoidberg_signal(:locked)
+        end
+        true
+      end
+
+      # Release the lock to access real instance
+      #
+      # @return [TrueClass]
+      def _release_lock!
+        if(@_lock)
+          if(@_locker == ::Thread.current)
+            @_locker_count -= 1
+            if(@_locker_count < 1)
+              @_locker = nil
+              @_lock.unlock if @_lock.locked?
+            end
+          end
+          _zoidberg_signal(:unlocked)
+        end
+        true
+      end
+
+      # Ensure any async threads are killed and accessing threads are
+      # forced into error state.
+      #
+      # @return [TrueClass]
+      def _zoidberg_destroy!(error=nil)
+        super do
+          _raw_threads[_raw_instance.object_id].map do |thread|
+            thread.raise ::Zoidberg::DeadException.new('Instance in terminated state!')
+          end.map do |thread|
+            thread.join(2)
+          end.find_all(&:alive?).map(&:kill)
+          _raw_threads.delete(_raw_instance.object_id)
+          @_accessing_threads.each do |thr|
+            if(thr.alive?)
+              begin
+                thr.raise ::Zoidberg::DeadException.new('Instance in terminated state!')
+              rescue
+              end
+            end
+          end
+          @_accessing_threads.clear
+        end
+      end
+
+    end
+
+
+  end
+
+end