concurrently 1.0.1

data/guides/Performance.md

@@ -0,0 +1,140 @@
+# Performance of Concurrently
+
+Overall, Concurrently is able to schedule around 100k to 200k concurrent
+evaluations per second. What to expect exactly is narrowed down in the
+following benchmarks.
+
+The measurements were executed with Ruby 2.4.1 on an Intel i7-5820K 3.3 GHz
+running Linux 4.10. Garbage collection was disabled.
+
+
+## Calling a (Concurrent) Proc
+
+This benchmark compares all `#call` methods of a concurrent proc and a regular
+proc. The mere invocation of the method is measured. The proc itself does
+nothing.
+
+    Benchmarked Code
+    ----------------
+      proc = proc{}
+      conproc = concurrent_proc{}
+      
+      while elapsed_seconds < 1
+        # CODE #
+      end
+    
+    Results
+    -------
+      # CODE #
+      proc.call:                5423106 executions in 1.0000 seconds
+      conproc.call:              662314 executions in 1.0000 seconds
+      conproc.call_nonblock:     769164 executions in 1.0000 seconds
+      conproc.call_detached:     269385 executions in 1.0000 seconds
+      conproc.call_and_forget:   306099 executions in 1.0000 seconds
+
+Explanation of the results:
+
+* The difference between a regular and a concurrent proc is caused by
+  concurrent procs being evaluated in a fiber and doing some bookkeeping.
+* Of the two methods evaluating the proc in the foreground `#call_nonblock`
+  is faster than `#call`, because the implementation of `#call` uses
+  `#call_nonblock` and does a little bit more on top.
+* Of the two methods evaluating the proc in the background, `#call_and_forget`
+  is faster because `#call_detached` additionally creates an evaluation
+  object.
+* Running concurrent procs in the background is considerably slower because
+  in this setup `#call_detached` and `#call_and_forget` cannot reuse fibers.
+  Their evaluation is merely scheduled and not started and concluded. This
+  would happen during the next iteration of the event loop. But since the
+  `while` loop never waits for something [the loop is never entered]
+  [Troubleshooting/A_concurrent_proc_is_scheduled_but_never_run].
+  All this leads to the creation of a new fiber for each evaluation. This is
+  responsible for the largest chunk of time needed during the measurement.
+
+You can run the benchmark yourself by running the [script][perf/concurrent_proc_calls.rb]:
+
+    $ perf/concurrent_proc_calls.rb
+
+
+## Scheduling (Concurrent) Procs
+
+This benchmark is closer to the real usage of Concurrently. It includes waiting
+inside a concurrent proc.
+
+    Benchmarked Code
+    ----------------
+      conproc = concurrent_proc{ wait 0 }
+      
+      while elapsed_seconds < 1
+        1.times{ # CODE # }
+        wait 0 # to enter the event loop
+      end
+    
+    Results
+    -------
+      # CODE #
+      conproc.call:               72444 executions in 1.0000 seconds
+      conproc.call_nonblock:     103468 executions in 1.0000 seconds
+      conproc.call_detached:     114882 executions in 1.0000 seconds
+      conproc.call_and_forget:   117425 executions in 1.0000 seconds
+
+Explanation of the results:
+
+* Because scheduling is now the dominant factor, there is a large drop in the
+  number of executions compared to just calling the procs. This makes the
+  number of executions when calling the proc in a non-blocking way comparable.
+* Calling the proc in a blocking manner with `#call` is costly. A lot of time
+  is spend waiting for the result.
+
+You can run the benchmark yourself by running the [script][perf/concurrent_proc_calls_awaiting.rb]:
+
+    $ perf/concurrent_proc_calls_awaiting.rb
+
+
+## Scheduling (Concurrent) Procs and Evaluating Them in Batches
+
+Additional to waiting inside a proc, it calls the proc 100 times at once. All
+100 evaluations will then be evaluated in one batch during the next iteration
+of the event loop.
+
+This is a simulation for a server receiving multiple messages during one
+iteration of the event loop and processing all of them in one go.
+
+    Benchmarked Code
+    ----------------
+      conproc = concurrent_proc{ wait 0 }
+      
+      while elapsed_seconds < 1
+        100.times{ # CODE # }
+        wait 0 # to enter the event loop
+      end
+    
+    Results
+    -------
+      # CODE #
+      conproc.call:               76300 executions in 1.0006 seconds
+      conproc.call_nonblock:     186200 executions in 1.0002 seconds
+      conproc.call_detached:     180200 executions in 1.0000 seconds
+      conproc.call_and_forget:   193500 executions in 1.0004 seconds
+
+
+Explanation of the results:
+
+* `#call` does not profit from batching due to is synchronizing nature.
+* The other methods show an increased throughput compared to running just a
+  single evaluation per event loop iteration.
+
+The result of this benchmark is the upper bound for how many concurrent
+evaluations Concurrently is able to run per second. The number of executions
+does not change much with a varying batch size. Larger batches (e.g. 200+)
+gradually start to get a bit slower. A batch of 1000 evaluations still handles
+around 140k executions.
+
+You can run the benchmark yourself by running the [script][perf/concurrent_proc_calls_awaiting.rb]:
+
+    $ perf/concurrent_proc_calls_awaiting.rb 100
+
+
+[perf/concurrent_proc_calls.rb]: https://github.com/christopheraue/m-ruby-concurrently/blob/master/perf/concurrent_proc_calls.rb
+[perf/concurrent_proc_calls_awaiting.rb]: https://github.com/christopheraue/m-ruby-concurrently/blob/master/perf/concurrent_proc_calls_awaiting.rb
+[Troubleshooting/A_concurrent_proc_is_scheduled_but_never_run]: http://www.rubydoc.info/github/christopheraue/m-ruby-concurrently/file/guides/Troubleshooting.md#A_concurrent_proc_is_scheduled_but_never_run

data/guides/Troubleshooting.md

@@ -0,0 +1,262 @@
+# Troubleshooting
+
+To get an idea about the inner workings of Concurrently have a look at the
+[Flow of control][] section in the overview.
+
+## A concurrent proc is scheduled but never run
+
+Consider the following script:
+
+```ruby
+#!/bin/env ruby
+
+concurrently do
+  puts "I will be forgotten, like tears in the rain."
+end
+
+puts "Unicorns!"
+```
+
+Running it will only print:
+
+```
+Unicorns!
+```
+
+`concurrently{}` is a shortcut for `concurrent_proc{}.call_and_forget`
+which in turn does not evaluate its code right away but schedules it to run
+during the next iteration of the event loop. But, since the root evaluation did
+not await anything the event loop has never been entered and the evaluation of
+the concurrent proc has never been started.
+
+A more subtle variation of this behavior occurs in the following scenario:
+
+```ruby
+#!/bin/env ruby
+
+concurrently do
+  puts "Unicorns!"
+  wait 2
+  puts "I will be forgotten, like tears in the rain."
+end
+
+wait 1
+```
+
+Running it will also only print:
+
+```
+Unicorns!
+```
+
+This time, the root evaluation does await something, namely the end of a one
+second time frame. Because of this, the evaluation of the `concurrently` block
+is indeed started and immediately waits for two seconds. After one second the
+root evaluation is resumed and exits. The `concurrently` block is never awoken
+again from its now eternal beauty sleep.
+
+## A call is blocking the entire execution.
+
+```ruby
+#!/bin/env ruby
+
+r,w = IO.pipe
+
+concurrently do
+  w.write 'Wake up!'
+end
+
+r.readpartial 32
+```
+
+Here, although we are practically waiting for `r` to be readable we do so in a
+blocking manner (`IO#readpartial` is blocking). This brings the whole process
+to a halt, the event loop will not be entered and the `concurrently` block will
+not be run. It will not be written to the pipe which in turn creates a nice
+deadlock.
+
+You can use blocking calls to deal with I/O. But you should await readiness of
+the IO before. If instead of just `r.readpartial 32` we write:
+
+```ruby
+r.await_readable
+r.readpartial 32
+```
+
+we suspend the root evaluation, switch to the event loop which runs the
+`concurrently` block and once there is something to read from `r` the root
+evaluation is resumed.
+
+This approach is not perfect. It is not very efficient if we do not need to
+await readability at all and could read from `r` immediately. But it is still
+better than blocking everything by default.
+
+The most efficient way is doing a non-blocking read and only await readability
+if it is not readable:
+
+```ruby
+begin
+  r.read_nonblock 32
+rescue IO::WaitReadable
+  r.await_readable
+  retry
+end
+```
+
+## The event loop is jammed by too many or too expensive evaluations
+
+Let's talk about a concurrent proc with an infinite loop:
+
+```ruby
+evaluation = concurrent_proc do
+  loop do
+    puts "To infinity! And beyond!"
+  end
+end.call_detached
+
+concurrently do
+  evaluation.conclude_to :cancelled
+end
+```
+
+When the concurrent proc is scheduled to run it runs and runs and runs and
+never finishes. The event loop is never entered again and the other concurrent
+proc concluding the evaluation is never started.
+
+A less extreme example is something like:
+
+```ruby
+concurrent_proc do
+  loop do
+    wait 0.1
+    puts "timer triggered at: #{Time.now.strftime('%H:%M:%S.%L')}"
+    concurrently do
+      sleep 1 # defers the entire event loop
+    end
+  end
+end.call
+
+# => timer triggered at: 16:08:17.704
+# => timer triggered at: 16:08:18.705
+# => timer triggered at: 16:08:19.705
+# => timer triggered at: 16:08:20.705
+# => timer triggered at: 16:08:21.706
+```
+
+This is a timer that is supposed to run every 0.1 seconds and creates another
+evaluation that takes a full second to complete. But since it takes so long the
+loop also only gets a chance to run every second leading to a delay of 0.9
+seconds between the time the timer is supposed to run and the time it actually
+ran.
+
+## Forking the process causes issues
+
+A fork inherits the main thread and with it the event loop with all its
+internal state from the parent. This is a problem since fibers created in
+the parent process cannot be resume in the forked process. Trying to do so
+raises a "fiber called across stack rewinding barrier" error. Also, we
+probably do not want to continue watching the parent's IOs.
+
+To fix this, the event loop has to be [reinitialized][Concurrently::EventLoop#reinitialize!]
+directly after forking:
+
+```ruby
+fork do
+  Concurrently::EventLoop.current.reinitialize!
+  # ...
+end
+
+# ...
+```
+
+While reinitializing the event loop clears its list of IOs watched for
+readiness, the IOs themselves are left untouched. You are responsible for
+managing IOs (e.g. closing them).
+
+## Errors tear down the event loop
+  
+Every concurrent proc rescues the following errors happening during its
+evaluation: `NoMemoryError`, `ScriptError`, `SecurityError`, `StandardError`
+and `SystemStackError`. These are all errors that should not have an immediate
+influence on other evaluations or the application as a whole. They will not
+leak to the event loop and will not tear it down.
+
+All other errors happening inside a concurrent proc *will* tear down the
+event loop. These error types are: `SignalException`, `SystemExit` and the
+general `Exception`. In such a case the event loop exits by raising a
+[Concurrently::Error][].
+
+If your application rescues the error when the event loop is teared down
+and continues running (irb does this, for example) it will do so with a
+[reinitialized event loop] [Concurrently::EventLoop#reinitialize!].
+
+## Using Plain Fibers
+
+In principle, you can safely use plain ruby fibers alongside concurrent procs.
+Just make sure you are exclusively operating on these fibers to not
+accidentally interfere with the fibers managed by Concurrently. Be
+especially careful with `Fiber.yield` and `Fiber.current` inside a concurrent
+proc.
+
+## Fiber-local variables are treated as thread-local
+
+In Ruby, `Thread#[]`, `#[]=`, `#key?` and `#keys` operate on variables local
+to the current fiber and not the current thread. This behavior is not noticed
+most of the time because people rarely work explicitly with fibers. Then, each
+thread has exactly one fiber and thread-local and fiber-local variables behave
+the same way.
+
+But if fibers come into play and a single thread starts switching between them,
+these methods cause errors instantly. Since Concurrently is built upon fibers
+it needs to sail around those issues. Most of the time the real intention is to
+set variables local to the current thread; just like the receiver of said
+methods suggests. For this reason, `Thread#[]`, `#[]=`, `#key?` and `#keys` are
+boldly redirected to `Thread#thread_variable_get`, `#thread_variable_set`,
+`#thread_variable?` and `#thread_variables`.
+
+If you belong to the ones using fibers with variables indeed intended to be
+fiber-local, you have two options: 1) Don't use Concurrently or 2) change all
+these fibers to concurrent procs and use their evaluation's [data store]
+[Concurrently::Proc::Evaluation#brackets] to store the variables.
+
+```ruby
+fiber = Fiber.new do
+  Thread.current[:key] = "I intend to be fiber-local!"
+  puts Thread.current[:key]
+end
+
+fiber.resume
+```
+
+becomes:
+
+```ruby
+conproc = concurrent_proc do
+  Concurrently::Evaluation.current[:key] = "I'm evaluation-local!"
+  puts Concurrently::Evaluation.current[:key]
+end
+
+conproc.call
+```
+
+## FiberError: mprotect failed
+
+Each concurrent evaluation runs in a fiber. If your application creates more
+concurrent evaluations than are concluded, more and more fibers need to be
+created. At some point the creation of additional fibers fails with
+"FiberError: mprotect failed". This is caused by hitting the limit for the the
+number of distinct memory maps a process can have. The corresponding linux
+kernel parameter is `/proc/sys/vm/max_map_count` and has default value of 64k.
+Each fiber creates two memory maps leading to a default maximum of around 30k
+fibers. To create more fibers the `max_map_count` needs to be increased.
+
+```
+$ sysctl -w vm.max_map_count=65530
+```
+
+See also: https://stackoverflow.com/a/11685165/3323185
+
+[Flow of control]: http://www.rubydoc.info/github/christopheraue/m-ruby-concurrently/file/guides/Overview.md#Flow+of+control
+[Concurrently::EventLoop#reinitialize!]: http://www.rubydoc.info/github/christopheraue/m-ruby-concurrently/Concurrently/EventLoop#reinitialize!-instance_method
+[Concurrently::Error]: http://www.rubydoc.info/github/christopheraue/m-ruby-concurrently/Concurrently/Error
+[Concurrently::Proc::Evaluation#brackets]: http://www.rubydoc.info/github/christopheraue/m-ruby-concurrently/Concurrently/Proc/Evaluation#%5B%5D-instance_method

data/lib/Ruby/concurrently.rb

@@ -0,0 +1,12 @@
+require "fiber"
+require "nio"
+require "hitimes"
+require "callbacks_attachable"
+
+root = File.dirname File.dirname File.dirname __FILE__
+files =
+  Dir[File.join(root, 'ext', 'all', '**', '*.rb')].sort +
+  Dir[File.join(root, 'ext', 'Ruby', '**', '*.rb')].sort +
+  Dir[File.join(root, 'lib', 'all', '**', '*.rb')].sort +
+  Dir[File.join(root, 'lib', 'Ruby', '**', '*.rb')].sort
+files.each{ |f| require f }

data/lib/Ruby/concurrently/error.rb

@@ -0,0 +1,4 @@
+module Concurrently
+  # Ruby has additional error classes
+  RESCUABLE_ERRORS << NoMemoryError << SecurityError
+end

data/lib/Ruby/concurrently/event_loop.rb

@@ -0,0 +1,24 @@
+module Concurrently
+  # @api ruby_patches
+  # @since 1.0.0
+  class EventLoop
+    # Attach an event loop to every thread in Ruby.
+    def self.current
+      Thread.current.__concurrently_event_loop__
+    end
+
+    # Use hitimes for a faster calculation of time intervals.
+    time_module = Module.new do
+      def reinitialize!
+        @clock = Hitimes::Interval.new.tap(&:start)
+        super
+      end
+
+      def lifetime
+        @clock.to_f
+      end
+    end
+
+    prepend time_module
+  end
+end

data/lib/Ruby/concurrently/event_loop/io_selector.rb

@@ -0,0 +1,38 @@
+module Concurrently
+  # @api private
+  # Let Ruby use nio to select IOs.
+  class EventLoop::IOSelector
+    def initialize(event_loop)
+      @run_queue = event_loop.run_queue
+      @selector = NIO::Selector.new
+    end
+
+    def awaiting?
+      not @selector.empty?
+    end
+
+    def await_reader(io, evaluation)
+      monitor = @selector.register(io, :r)
+      monitor.value = evaluation
+    end
+
+    def await_writer(io, evaluation)
+      monitor = @selector.register(io, :w)
+      monitor.value = evaluation
+    end
+
+    def cancel_reader(io)
+      @selector.deregister(io)
+    end
+
+    def cancel_writer(io)
+      @selector.deregister(io)
+    end
+
+    def process_ready_in(waiting_time)
+      @selector.select(waiting_time) do |monitor|
+        @run_queue.resume_evaluation! monitor.value, true
+      end
+    end
+  end
+end