eventbox 0.1.0

data/Rakefile

@@ -0,0 +1,14 @@
+require "bundler/gem_tasks"
+require "rake/testtask"
+require "yard"
+
+Rake::TestTask.new(:test) do |t|
+  t.libs << "test"
+  t.libs << "lib"
+  t.test_files = FileList["test/**/*_test.rb"]
+end
+
+YARD::Rake::YardocTask.new do |t|
+end
+
+task :gem => :build

data/bin/console

@@ -0,0 +1,14 @@
+#!/usr/bin/env ruby
+
+require "bundler/setup"
+require "eventbox"
+
+# You can add fixtures and/or initialization code here to make experimenting
+# with your gem easier. You can also use a different console, if you like.
+
+# (If you use this, don't forget to add pry to your Gemfile!)
+# require "pry"
+# Pry.start
+
+require "irb"
+IRB.start(__FILE__)

data/bin/setup

@@ -0,0 +1,8 @@
+#!/usr/bin/env bash
+set -euo pipefail
+IFS=$'\n\t'
+set -vx
+
+bundle install
+
+# Do any other automated setup that you need to do here

data/docs/downloads.md

@@ -0,0 +1,143 @@
+### Use Eventbox to download URLs concurrently
+
+The following example illustrates how to use actions in order to download a list of URLs in parallel.
+
+At first the `init` method starts an action for each URL to be downloaded, initializes some variables and stores the `result` object for later use.
+Since the `result` is not yielded in the method body, the external call to `ParallelDownloads.new` doesn't return to that point in time.
+Instead it's suspended until `result` is yielded later on, when all URLs have been retrieved.
+
+### Running actions
+
+Each call to the action method `start_download` starts a new thread (or at least borrows one from the thread-pool).
+That way we leave the protected event scope of {Eventbox.async_call async_call}, {Eventbox.sync_call sync_call} and {Eventbox.yield_call yield_call} methods and enter the action scope which runs concurrently.
+Since actions don't have access to instance variables, all required information must be passed as method arguments.
+This is intentionally, because all arguments pass the {Eventbox::Sanitizer} that way, which protects from data races and translates between internal event based and external blocking behavior of `Proc` objects.
+Actions should never use shared data directly or share any data with other program parts, but should use event scope methods like {Eventbox.sync_call sync_call} or closures like {Eventbox#yield_proc yield_proc} to access shared data in a thread-safe way.
+
+### Catching errors
+
+Another typical and recommended code sequence is the `rescue` / `else` declaration in an action method.
+They inform the Eventbox object about success or failure of a particular action.
+This outcome can then be properly handled by event scope methods.
+In our case either the received data or the received exception is sent to `download_finished`.
+It is a event scope method, so that it can safely access instance variables.
+If all downloads completed, the result object received at `init` is yielded, so that the external call to `ParallelDownloads.new` returns.
+
+Let's see how this looks in practice:
+
+```ruby
+require "eventbox"
+require "net/https"
+require "open-uri"
+require "pp"
+
+# Build a new Eventbox based class, which makes use of a pool of two threads.
+# This way the number of concurrent downloads is limited to 3.
+class ParallelDownloads < Eventbox.with_options(threadpool: Eventbox::ThreadPool.new(3))
+
+  # Called at ParallelDownloads.new just like Object#initialize in ordinary ruby classes
+  # Yield calls get one additional argument and suspend the caller until result.yield is invoked
+  yield_call def init(urls, result, &progress)
+    @urls = urls
+    @urls.each do |url|             # Start a download thread for each URL
+      start_download(url)           # Start the download - the call returns immediately
+    end
+    # It's safe to set instance variables after start_download
+    @downloads = {}                 # The result hash with all downloads
+    @finished = result              # Don't return to the caller, but store result yielder for later
+    @progress = progress
+  end
+
+  # Each call to an action method starts a new thread
+  # Actions don't have access to instance variables.
+  private action def start_download(url)
+    data = OpenURI.open_uri(url)    # HTTP GET url
+      .read(100).each_line.first    # Retrieve the first line but max 100 bytes
+  rescue SocketError => err         # Catch any network errors
+    download_finished(url, err)     # and store it in the result hash
+  else
+    download_finished(url, data)    # ... or store the retrieved data when successful
+  end
+
+  # Called for each finished download
+  private sync_call def download_finished(url, res)
+    @downloads[url] = res             # Store the download result in the result hash
+    @progress&.yield(@downloads.size) # Notify the caller about our progress
+    if @downloads.size == @urls.size  # All downloads finished?
+      @finished.yield                 # Finish ParallelDownloads.new
+    end
+  end
+
+  attr_reader :downloads            # Threadsafe access to @download
+end
+
+urls = %w[
+  http://ruby-lang.org
+  http://ruby-lang.ooorg
+  http://wikipedia.org
+  http://torproject.org
+  http://github.com
+]
+
+d = ParallelDownloads.new(urls) { |progress| print progress }
+pp d.downloads
+```
+
+This prints the numbers 1 to 5 as downloads finish and subsequently prints the reveived HTML text, so that the output looks like the following.
+The order depends on the particular response time of the URL.
+
+```ruby
+12345{"http://ruby-lang.ooorg"=>#<SocketError: Failed to open TCP connection to ruby-lang.ooorg:80 (getaddrinfo: Name or service not known)>,
+ "http://wikipedia.org"=>"<!DOCTYPE html>\n",
+ "http://torproject.org"=>"<div class=\"eoy-background\">\n",
+ "http://ruby-lang.org"=>"<!DOCTYPE html>\n",
+ "http://github.com"=>"\n"}
+```
+
+Since Eventbox protects from data races, it's insignificant in which order events are emitted by an event scope method and whether objects are changed after being sent.
+It's therefore OK to set `@downloads` both before or after starting the action threads per `start_download` in `init`.
+
+### Change to closure style
+
+There is another alternative way to transmit the result of an action to the event scope.
+Instead of calling a {Eventbox.sync_call sync_call} method a closure like {Eventbox.sync_proc sync_proc} can be used.
+It is simply the anonymous form of {Eventbox.sync_call sync_call}.
+It behaves exactly identical, but is passed as argument.
+This means in particular, that it's thread-safe to call {Eventbox.sync_proc sync_proc} from an action or external scope.
+
+The above class rewritten to the closure style looks like so:
+
+```ruby
+class ParallelDownloads < Eventbox.with_options(threadpool: Eventbox::ThreadPool.new(3))
+
+  yield_call def init(urls, result, &progress)
+    urls.each do |url|                   # Start a download thread for each URL
+
+      on_finished = sync_proc do |res|   # Create a closure object comparable to sync_call
+        @downloads[url] = res            # Store the download result in the result hash
+        progress&.yield(@downloads.size) # Notify the caller about our progress
+        if @downloads.size == urls.size  # All downloads finished?
+          result.yield                   # Let ParallelDownloads.new return
+        end
+      end
+
+      start_download(url, on_finished)   # Start the download - the call returns immediately
+    end
+    @downloads = {}                      # The result hash with all downloads
+  end
+
+  private action def start_download(url, on_finished)
+    data = OpenURI.open_uri(url)         # HTTP GET url
+      .read(100).each_line.first         # Retrieve the first line but max 100 bytes
+  rescue SocketError => err              # Catch any network errors
+    on_finished.yield(err)               # and store it in the result hash
+  else
+    on_finished.yield(data)              # ... or store the retrieved data when successful
+  end
+
+  attr_reader :downloads                 # Threadsafe access to @download
+end
+```
+
+I guess that friends of object orientated programming probably like the method style more, while fans of functional programming prefer closures.
+All in all it's purely a matter of taste whether you prefer the method or the closure style.

data/docs/server.md

@@ -0,0 +1,88 @@
+Race-free server startup and shutdown can be a tricky task.
+The following example illustrates, how a TCP server can be started and interrupted properly.
+
+```ruby
+require "eventbox"
+require "socket"
+
+class MyServer < Eventbox
+  yield_call def init(bind, port, result)
+    @count = 0
+    @server = start_serving(bind, port, result)
+  end
+
+  action def start_serving(bind, port, init_done)
+    serv = TCPServer.new(bind, port)
+  rescue => err
+    init_done.raise err
+  else
+    init_done.yield
+
+    loop do
+      begin
+        conn = Thread.handle_interrupt(Stop => :on_blocking) do
+          serv.accept
+        end
+      rescue Stop => st
+        serv.close
+        st.stopped.yield
+        break
+      else
+        MyConnection.new(conn, self)
+      end
+    end
+  end
+
+  sync_call def count
+    @count += 1
+  end
+
+  yield_call def stop(result)
+    @server.raise(Stop.new(result))
+  end
+
+  class Stop < RuntimeError
+    def initialize(stopped)
+      @stopped = stopped
+    end
+    attr_reader :stopped
+  end
+end
+
+class MyConnection < Eventbox
+  action def init(conn, server)
+    conn.write "Hello #{server.count}"
+  ensure
+    conn.close
+  end
+end
+```
+
+The server can now be started like so.
+
+```ruby
+s = MyServer.new('localhost', 12345)
+
+10.times.map do
+  Thread.new do
+    TCPSocket.new('localhost', 12345).read
+  end
+end.each { |th| p th.value }
+
+s.stop
+```
+
+It prints some output like this:
+
+```ruby
+"Hello 2"
+"Hello 1"
+"Hello 7"
+"Hello 8"
+"Hello 3"
+"Hello 9"
+"Hello 5"
+"Hello 6"
+"Hello 4"
+"Hello 10"
+```

data/docs/threadpool.md

@@ -0,0 +1,73 @@
+The following class implements a thread-pool with a fixed number of threads to be borrowed by the `pool` method.
+It shows how the action method `start_pool_thread` makes use of the private yield_call `next_job` to query, wait for and retrieve an object from the event scope.
+
+This kind of object is the block that is given to `pool`.
+Although all closures (blocks, procs and lambdas) are wrapped in a way that allows safe calls from the event scope, it is just passed through to the action scope and retrieved as the result value of `next_job`.
+When this happens, the wrapping is automatically removed, so that the pure block given to `pool` is called in `start_pool_thread`.
+
+```ruby
+class ThreadPool < Eventbox
+  async_call def init(pool_size)
+    @que = []                 # Initialize an empty job queue
+    @jobless = []             # Initialize the list of jobless action threads
+
+    pool_size.times do        # Start up x action threads
+      start_pool_thread
+    end
+  end
+
+  # The action call returns immediately, but spawns a new thread.
+  private action def start_pool_thread
+    while bl=next_job     # Each new thread waits for a job to be pooled
+      bl.call             # Execute the external job enqueued by `pool`
+    end
+  end
+
+  # Get the next job or wait for one
+  # The method is private, so that it's accessible in start_pool_thread action but not externally
+  private yield_call def next_job(result)
+    if @que.empty?            # No job pooled?
+      @jobless << result      # Enqueue the action thread to the list of jobless workers
+    else                      # Already pooled jobs?
+      result.yield @que.shift # Take the oldest job and let next_job return with this job
+    end
+  end
+
+  # Enqueue a new job
+  async_call def pool(&block)
+    if @jobless.empty?        # No jobless thread available?
+      @que << block           # Append the external block as job into the queue
+    else                      # A thread is waiting?
+      @jobless.shift.yield block  # Take one thread and let next_job return the given job
+    end                           # so that it is processed by the pool_thread action above
+  end
+end
+```
+
+This `ThreadPool` can be used like so:
+
+```ruby
+tp = ThreadPool.new(3)  # Create a thread pool with 3 action threads
+5.times do |i|          # Start 5 jobs concurrently
+  tp.pool do            # pool never blocks, but enqueues jobs when no free thread is available
+    sleep 1             # The mission of each job: Wait for 1 second (3 jobs concurrently)
+    p [i, Thread.current.object_id]
+  end
+end
+
+# It gives something like the following output after 1 second:
+[2, 47030774465880]
+[1, 47030775602740]
+[0, 47030774464940]
+# and something like this after one more seconds:
+[3, 47030775602740]
+[4, 47030774465880]
+```
+
+Eventbox's builtin thread-pool {Eventbox::ThreadPool} is implemented on top of Eventbox similar to the above.
+In addition there are various battle proof implementations of thread-pools such a these in [concurrent-ruby](https://github.com/ruby-concurrency/concurrent-ruby), which are faster and more feature rich than the above.
+
+However Eventbox comes into play when things are getting more complicated or more customized.
+Imagine the thread-pool has to schedule it's tasks not just to cheep threads, but to more expensive or more constraint resources.
+In such cases available abstractions don't fit well to the problem.
+Instead the above example can be used as a basis for your own extensions.

data/eventbox.gemspec

@@ -0,0 +1,31 @@
+lib = File.expand_path("../lib", __FILE__)
+$LOAD_PATH.unshift(lib) unless $LOAD_PATH.include?(lib)
+require "eventbox/version"
+
+Gem::Specification.new do |spec|
+  spec.name          = "eventbox"
+  spec.version       = Eventbox::VERSION
+  spec.authors       = ["Lars Kanis"]
+  spec.email         = ["lars@greiz-reinsdorf.de"]
+
+  if File.read("README.md", encoding: 'utf-8') =~ /^_(.*?)_$\s^\n(.*?)\n$/m
+    spec.summary = $1
+    spec.description = $2
+  end
+  spec.homepage      = "https://github.com/larskanis/eventbox"
+  spec.license       = "MIT"
+
+  spec.files         = Dir.chdir(File.expand_path('..', __FILE__)) do
+    `git ls-files -z`.split("\x0").reject { |f| f.match(%r{^(test)/}) }
+  end
+  spec.bindir        = "exe"
+  spec.executables   = spec.files.grep(%r{^exe/}) { |f| File.basename(f) }
+  spec.require_paths = ["lib"]
+  spec.required_ruby_version = "~> 2.3"
+  spec.metadata["yard.run"] = "yri" # use "yard" to build full HTML docs.
+
+  spec.add_development_dependency "bundler", ">= 1.16", "< 3"
+  spec.add_development_dependency "rake", "~> 10.0"
+  spec.add_development_dependency "minitest", "~> 5.0"
+  spec.add_development_dependency "yard", "~> 0.9"
+end

data/lib/eventbox.rb

@@ -0,0 +1,270 @@
+# frozen-string-literal: true
+
+require "weakref"
+require "eventbox/argument_wrapper"
+require "eventbox/sanitizer"
+require "eventbox/boxable"
+require "eventbox/event_loop"
+require "eventbox/object_registry"
+
+class Eventbox
+  autoload :VERSION, "eventbox/version"
+  autoload :ThreadPool, "eventbox/thread_pool"
+  autoload :Timer, "eventbox/timer"
+
+  extend Boxable
+
+  class InvalidAccess < RuntimeError; end
+  class MultipleResults < RuntimeError; end
+  class AbortAction < RuntimeError; end
+
+  if RUBY_ENGINE=='jruby' && RUBY_VERSION.split(".").map(&:to_i).pack("C*") < [9,2,1,0].pack("C*") ||
+      RUBY_ENGINE=='truffleruby'
+    # This is a workaround for bug https://github.com/jruby/jruby/issues/5314
+    # which was fixed in JRuby-9.2.1.0.
+    class Thread < ::Thread
+      def initialize(*args, &block)
+        started = Queue.new
+        super do
+          Thread.handle_interrupt(Exception => :never) do
+            started << true
+            block.call(*args)
+            # Immediately stop the thread, before the handle_interrupt has finished.
+            # This is necessary for JRuby to avoid possoble signal handling after the block.
+            Thread.exit
+          end
+        end
+        started.pop
+      end
+    end
+  end
+
+  # Retrieves the Eventbox options of this class.
+  #
+  # @return [Hash]  The options for instantiation of this class.
+  # @see with_options
+  def self.eventbox_options
+    {
+      threadpool: Thread,
+      guard_time: 0.5,
+      gc_actions: false,
+    }
+  end
+
+  # Create a new derived class with the given options.
+  #
+  # The options are merged with the options of the base class.
+  # The following options are available:
+  #
+  # @param threadpool [Object] A threadpool.
+  #   Can be either +Thread+ (default) or a {Eventbox::Threadpool} instance.
+  # @param guard_time Event scope methods should not do blocking operations.
+  #   Eventbox measures the time of each call to event scope methods and warns, when it is exceeded.
+  #   There are several ways to configure guard_time:
+  #   * Set to +nil+: Disable measuring of time to process event scope methods.
+  #   * Set to a +Numeric+ value: Maximum number of seconds allowed for event scope methods.
+  #   * Set to a +Proc+ object: Called after each call to an event scope method.
+  #     The +Proc+ object is called with the number of seconds the call took as first and the name as second argument.
+  # @param gc_actions [Boolean] Enable or disable (default) garbage collection of running actions.
+  #   Setting this to true permits the garbage collector to shutdown running action threads and subsequently delete the corresponding Eventbox object.
+  def self.with_options(**options)
+    Class.new(self) do
+      define_singleton_method(:eventbox_options) do
+        super().merge(options)
+      end
+
+      def self.inspect
+        klazz = self
+        until name=klazz.name
+          klazz = klazz.superclass
+        end
+        "#{name}#{eventbox_options}"
+      end
+    end
+  end
+
+  private
+
+  # @private
+  #
+  # Create a new {Eventbox} instance.
+  #
+  # All arguments are passed to the init() method when defined.
+  def initialize(*args, &block)
+    options = self.class.eventbox_options
+
+    # This instance variable is set to self here, but replaced by Boxable#action to a WeakRef
+    @__eventbox__ = self
+
+    # Verify that all public methods are properly wrapped and no unsafe methods exist
+    # This check is done at the first instanciation only and doesn't slow down subsequently.
+    # Since test and set operations aren't atomic, it can happen that the check is executed several times.
+    # This is considered less harmful than slowing all instanciations down by a mutex.
+    unless self.class.instance_variable_defined?(:@eventbox_methods_checked)
+      self.class.instance_variable_set(:@eventbox_methods_checked, true)
+
+      obj = Object.new
+      meths = methods - obj.methods - [:__getobj__, :shutdown!, :shared_object]
+      prmeths = private_methods - obj.private_methods
+      prohib = meths.find do |name|
+        !prmeths.include?(:"__#{name}__")
+      end
+      if prohib
+        meth = method(prohib)
+        raise InvalidAccess, "method `#{prohib}' at #{meth.source_location.join(":")} is not properly defined -> it must be created per async_call, sync_call, yield_call or private prefix"
+      end
+    end
+
+    # Run the processing of calls (the event loop) in a separate class.
+    # Otherwise it would block GC'ing of self.
+    @__event_loop__ = EventLoop.new(options[:threadpool], options[:guard_time])
+    ObjectSpace.define_finalizer(self, @__event_loop__.method(:send_shutdown))
+
+    init(*args, &block)
+  end
+
+  def self.method_added(name)
+    if name==:initialize
+      meth = instance_method(:initialize)
+      raise InvalidAccess, "method `initialize' at #{meth.source_location.join(":")} must not be overwritten - use `init' instead"
+    end
+  end
+
+  # @private
+  #
+  # Provide access to the eventbox instance as either
+  # - self within the eventbox instance itself or
+  # - WeakRef.new(self).__getobj__ within actions.
+  # This allows actions to be GC'ed, when the related Eventbox instance is no longer in use.
+  def eventbox
+    @__eventbox__.__getobj__
+  end
+
+  # @private
+  protected def __getobj__
+    self
+  end
+
+  private
+
+  # Initialize a new {Eventbox} instance.
+  #
+  # This method is executed for initialization of a Eventbox instance.
+  # This method receives all arguments given to +Eventbox.new+ after they have passed the {Sanitizer}.
+  # It can be used like +initialize+ in ordinary ruby classes including +super+ to initialize included modules or base classes.
+  #
+  # {init} can be defined as either {sync_call} or {async_call} with no difference.
+  # {init} can also be defined as {yield_call}, so that the +new+ call is blocked until the result is yielded.
+  # {init} can even be defined as {action}, so that each instance of the class immediately starts a new thread.
+  def init(*args)
+  end
+
+  # Create a proc object for asynchronous (fire-and-forget) calls similar to {async_call}.
+  #
+  # It can be passed to external scope and called from there like so:
+  #
+  #   class MyBox < Eventbox
+  #     sync_call def print(p1)
+  #       async_proc do |p2|
+  #         puts "#{p1} #{p2}"
+  #       end
+  #     end
+  #   end
+  #   MyBox.new.print("Hello").call("world")   # Prints "Hello world"
+  #
+  # The created object can be safely called from any thread.
+  # All block arguments are passed through the {Sanitizer}.
+  # The block itself might not do any blocking calls or expensive computations - this would impair responsiveness of the {Eventbox} instance.
+  # Instead use {Eventbox.action} in these cases.
+  #
+  # The block always returns +self+ to the caller.
+  def async_proc(name=nil, &block)
+    @__event_loop__.new_async_proc(name=nil, &block)
+  end
+
+  # Create a Proc object for synchronous calls similar to {sync_call}.
+  #
+  # It can be passed to external scope and called from there like so:
+  #
+  #   class MyBox < Eventbox
+  #     sync_call def print(p1)
+  #       sync_proc do |p2|
+  #         "#{p1} #{p2}"
+  #       end
+  #     end
+  #   end
+  #   puts MyBox.new.print("Hello").call("world")   # Prints "Hello world"
+  #
+  # The created object can be safely called from any thread.
+  # All block arguments as well as the result value are passed through the {Sanitizer}.
+  # The block itself might not do any blocking calls or expensive computations - this would impair responsiveness of the {Eventbox} instance.
+  # Instead use {Eventbox.action} in these cases.
+  #
+  # This Proc is simular to {async_proc}, but when the block is invoked, it is executed and it's return value is returned to the caller.
+  # Since all processing within the event scope of an {Eventbox} instance must not execute blocking operations, sync procs can only return immediate values.
+  # For deferred results use {yield_proc} instead.
+  def sync_proc(name=nil, &block)
+    @__event_loop__.new_sync_proc(name=nil, &block)
+  end
+
+  # Create a Proc object for calls with deferred result similar to {yield_call}.
+  #
+  # It can be passed to external scope and called from there like so:
+  #
+  #   class MyBox < Eventbox
+  #     sync_call def print(p1)
+  #       yield_proc do |p2, result|
+  #         result.yield "#{p1} #{p2}"
+  #       end
+  #     end
+  #   end
+  #   puts MyBox.new.print("Hello").call("world")   # Prints "Hello world"
+  #
+  # This proc type is simular to {sync_proc}, however it's not the result of the block that is returned.
+  # Instead the block is called with one additional argument in the event scope, which is used to yield a result value.
+  # The result value can be yielded within the called block, but it can also be called by any other event scope or external method, leading to a deferred proc return.
+  # The external thread calling this proc is suspended until a result is yielded.
+  # However the Eventbox object keeps responsive to calls from other threads.
+  #
+  # The created object can be safely called from any thread.
+  # If yield procs are called in the event scope, they must get a Proc object as the last argument.
+  # It is called when a result was yielded.
+  #
+  # All block arguments as well as the result value are passed through the {Sanitizer}.
+  # The block itself might not do any blocking calls or expensive computations - this would impair responsiveness of the {Eventbox} instance.
+  # Instead use {Eventbox.action} in these cases.
+  def yield_proc(name=nil, &block)
+    @__event_loop__.new_yield_proc(name=nil, &block)
+  end
+
+  # Mark an object as to be shared instead of copied.
+  #
+  # A marked object is never passed as copy, but passed as reference.
+  # The object is therefore wrapped as {WrappedObject} when used in an unsafe scope.
+  # Wrapping as {WrappedObject} denies access from external/action scope to event scope objects and vice versa.
+  # It also denies access to objects originated from a foreign event scope.
+  # However the object can be passed as reference and is automatically unwrapped when passed back to the original scope.
+  # It can therefore be used to modify the original object even after traversing the boundaries.
+  #
+  # Wrapping and unwrapping works even if the shared object is stored within another object as instance variable or within a collection class.
+  #
+  # The mark is stored for the lifetime of the object, so that it's enough to mark only once at object creation.
+  public def shared_object(object)
+    @__event_loop__.shared_object(object)
+  end
+
+  # Force stop of all action threads spawned by this {Eventbox} instance.
+  #
+  # It is possible to enable automatic cleanup of action threads by the garbage collector through {Eventbox.with_options}.
+  # However in some cases automatic garbage collection doesn't remove all instances due to running action threads.
+  # Calling shutdown! when the work of the instance is done, ensures that it is GC'ed in all cases.
+  #
+  # If {shutdown!} is called externally, it blocks until all actions threads have terminated.
+  #
+  # If {shutdown!} is called in the event scope, it just triggers the termination of all action threads and returns afterwards.
+  # Optionally {shutdown!} can be called with a block.
+  # It is called when all actions threads terminated.
+  public def shutdown!(&completion_block)
+    @__event_loop__.shutdown(&completion_block)
+  end
+end