concurrently 1.0.1 → 1.1.0

data/guides/Performance.md

@@ -1,140 +1,338 @@
 # Performance of Concurrently
 
-Overall, Concurrently is able to schedule around 100k to 200k concurrent
-evaluations per second. What to expect exactly is narrowed down in the
-following benchmarks.
+The measurements were executed on an Intel i7-5820K 3.3 GHz running Linux 4.10.
+Garbage collection was disabled. The benchmark runs the code in batches to
+reduce the overhead of the benchmark harness.
 
-The measurements were executed with Ruby 2.4.1 on an Intel i7-5820K 3.3 GHz
-running Linux 4.10. Garbage collection was disabled.
+## Mere Invocation of Concurrent Procs
 
+This benchmark compares all call methods of a [concurrent proc][Concurrently::Proc] 
+and a regular proc. The procs itself do nothing. The results represent the
+baseline for how fast Concurrently is able to work. It can't get any faster
+than that.
 
-## Calling a (Concurrent) Proc
-
-This benchmark compares all `#call` methods of a concurrent proc and a regular
-proc. The mere invocation of the method is measured. The proc itself does
-nothing.
-
-    Benchmarked Code
-    ----------------
-      proc = proc{}
-      conproc = concurrent_proc{}
-      
-      while elapsed_seconds < 1
-        # CODE #
-      end
+    Benchmarks
+    ----------
+      proc.call:
+        test_proc = proc{}
+        batch = Array.new(100)
+        
+        while elapsed_seconds < 1
+          batch.each{ test_proc.call }
+        end
+        
+      conproc.call:
+        test_proc = concurrent_proc{}
+        batch = Array.new(100)
+        
+        while elapsed_seconds < 1
+          batch.each{ test_proc.call }
+        end
+        
+      conproc.call_nonblock:
+        test_proc = concurrent_proc{}
+        batch = Array.new(100)
+        
+        while elapsed_seconds < 1
+          batch.each{ test_proc.call_nonblock }
+        end
+        
+      conproc.call_detached:
+        test_proc = concurrent_proc{}
+        batch = Array.new(100)
+        
+        while elapsed_seconds < 1
+          batch.each{ test_proc.call_detached }
+          wait 0
+        end
+        
+      conproc.call_and_forget:
+        test_proc = concurrent_proc{}
+        batch = Array.new(100)
+        
+        while elapsed_seconds < 1
+          batch.each{ test_proc.call_and_forget }
+          wait 0
+        end
+        
+    Results for ruby 2.4.1
+    ----------------------
+      proc.call:                11048400 executions in 1.0000 seconds
+      conproc.call:               734000 executions in 1.0000 seconds
+      conproc.call_nonblock:      857800 executions in 1.0001 seconds
+      conproc.call_detached:      464800 executions in 1.0002 seconds
+      conproc.call_and_forget:    721800 executions in 1.0001 seconds
     
-    Results
-    -------
-      # CODE #
-      proc.call:                5423106 executions in 1.0000 seconds
-      conproc.call:              662314 executions in 1.0000 seconds
-      conproc.call_nonblock:     769164 executions in 1.0000 seconds
-      conproc.call_detached:     269385 executions in 1.0000 seconds
-      conproc.call_and_forget:   306099 executions in 1.0000 seconds
+    Results for mruby 1.3.0
+    -----------------------
+      proc.call:                 4771700 executions in 1.0000 seconds
+      conproc.call:               362000 executions in 1.0002 seconds
+      conproc.call_nonblock:      427400 executions in 1.0000 seconds
+      conproc.call_detached:      188900 executions in 1.0005 seconds
+      conproc.call_and_forget:    383400 executions in 1.0002 seconds
+
+*conproc.call_detached* and *conproc.call_and_forget* call `wait 0` after each
+batch so the scheduled evaluations have [a chance to run]
+[Troubleshooting/A_concurrent_proc_is_scheduled_but_never_run]. Otherwise,
+their evaluations were merely scheduled and not started and concluded like it
+is happening in the other cases. This makes the benchmarks comparable.
 
 Explanation of the results:
 
 * The difference between a regular and a concurrent proc is caused by
   concurrent procs being evaluated in a fiber and doing some bookkeeping.
-* Of the two methods evaluating the proc in the foreground `#call_nonblock`
-  is faster than `#call`, because the implementation of `#call` uses
-  `#call_nonblock` and does a little bit more on top.
-* Of the two methods evaluating the proc in the background, `#call_and_forget`
-  is faster because `#call_detached` additionally creates an evaluation
+* Of the two methods evaluating the proc in the foreground
+  [Concurrently::Proc#call_nonblock][] is faster than [Concurrently::Proc#call][],
+  because the implementation of [Concurrently::Proc#call][] uses
+  [Concurrently::Proc#call_nonblock][] and does a little bit more on top.
+* Of the two methods evaluating the proc in the background,
+  [Concurrently::Proc#call_and_forget][] is faster because
+  [Concurrently::Proc#call_detached][] additionally creates an evaluation
   object.
-* Running concurrent procs in the background is considerably slower because
-  in this setup `#call_detached` and `#call_and_forget` cannot reuse fibers.
-  Their evaluation is merely scheduled and not started and concluded. This
-  would happen during the next iteration of the event loop. But since the
-  `while` loop never waits for something [the loop is never entered]
-  [Troubleshooting/A_concurrent_proc_is_scheduled_but_never_run].
-  All this leads to the creation of a new fiber for each evaluation. This is
-  responsible for the largest chunk of time needed during the measurement.
+* Running concurrent procs in the background is slower than running them in the
+  foreground because their evaluations need to be scheduled.
+* Overall, mruby is about half as fast as Ruby.
+
+You can run this benchmark yourself by executing:
 
-You can run the benchmark yourself by running the [script][perf/concurrent_proc_calls.rb]:
+    $ rake benchmark[call_methods]
 
-    $ perf/concurrent_proc_calls.rb
 
+## Mere Waiting
 
-## Scheduling (Concurrent) Procs
+This benchmark measures the number of times per second we can
 
-This benchmark is closer to the real usage of Concurrently. It includes waiting
-inside a concurrent proc.
+* wait an amount of time,
+* await readability of an IO object and
+* await writability of an IO object.
 
-    Benchmarked Code
-    ----------------
-      conproc = concurrent_proc{ wait 0 }
-      
-      while elapsed_seconds < 1
-        1.times{ # CODE # }
-        wait 0 # to enter the event loop
-      end
+Like with calling a proc doing nothing this defines what maximum performance
+to expect in these cases.
+
+    Benchmarks
+    ----------
+      wait:
+        test_proc = proc do
+          wait 0 # schedule the proc be resumed ASAP
+        end
+        
+        batch = Array.new(100)
+        
+        while elapsed_seconds < 1
+          batch.each{ test_proc.call }
+        end
+        
+      await_readable:
+        test_proc = proc do |r,w|
+          r.await_readable
+        end
+        
+        batch = Array.new(100) do |idx|
+          IO.pipe.tap{ |r,w| w.write '0' }
+        end
+        
+        while elapsed_seconds < 1
+          batch.each{ |*args| test_proc.call(*args) }
+        end
+        
+      await_writable:
+        test_proc = proc do |r,w|
+          w.await_writable
+        end
+        
+        batch = Array.new(100) do |idx|
+          IO.pipe
+        end
+        
+        while elapsed_seconds < 1
+          batch.each{ |*args| test_proc.call(*args) }
+        end
+        
+    Results for ruby 2.4.1
+    ----------------------
+      wait:                       291100 executions in 1.0001 seconds
+      await_readable:             147800 executions in 1.0005 seconds
+      await_writable:             148300 executions in 1.0003 seconds
     
-    Results
-    -------
-      # CODE #
-      conproc.call:               72444 executions in 1.0000 seconds
-      conproc.call_nonblock:     103468 executions in 1.0000 seconds
-      conproc.call_detached:     114882 executions in 1.0000 seconds
-      conproc.call_and_forget:   117425 executions in 1.0000 seconds
+    Results for mruby 1.3.0
+    -----------------------
+      wait:                       104300 executions in 1.0002 seconds
+      await_readable:             132600 executions in 1.0006 seconds
+      await_writable:             130500 executions in 1.0005 seconds
 
 Explanation of the results:
 
-* Because scheduling is now the dominant factor, there is a large drop in the
-  number of executions compared to just calling the procs. This makes the
-  number of executions when calling the proc in a non-blocking way comparable.
-* Calling the proc in a blocking manner with `#call` is costly. A lot of time
-  is spend waiting for the result.
-
-You can run the benchmark yourself by running the [script][perf/concurrent_proc_calls_awaiting.rb]:
+* In Ruby, waiting an amount of time is much faster than awaiting readiness of
+  I/O because it does not need to enter the underlying poll call.
+* In mruby, awaiting readiness of I/O is actually faster than just waiting an
+  amount of time. Scheduling an evaluation to resume at a specific time
+  involves amongst other things inserting it into an array at the right index.
+  mruby implements many Array methods in plain ruby which makes them noticeably
+  slower.
 
-    $ perf/concurrent_proc_calls_awaiting.rb
+You can run this benchmark yourself by executing:
 
+    $ rake benchmark[wait_methods]
 
-## Scheduling (Concurrent) Procs and Evaluating Them in Batches
 
-Additional to waiting inside a proc, it calls the proc 100 times at once. All
-100 evaluations will then be evaluated in one batch during the next iteration
-of the event loop.
+## Waiting Inside Concurrent Procs
 
-This is a simulation for a server receiving multiple messages during one
-iteration of the event loop and processing all of them in one go.
+Concurrent procs show different performance depending on how they are called
+and if their evaluation needs to wait or not. This benchmark explores these
+differences and serves as a guide which call method provides the best
+performance in these scenarios.
 
-    Benchmarked Code
-    ----------------
-      conproc = concurrent_proc{ wait 0 }
-      
-      while elapsed_seconds < 1
-        100.times{ # CODE # }
-        wait 0 # to enter the event loop
-      end
+    Benchmarks
+    ----------
+      call:
+        test_proc = concurrent_proc{}
+        batch = Array.new(100)
+        
+        while elapsed_seconds < 1
+          batch.each{ test_proc.call }
+          # Concurrently::Proc#call already synchronizes the results of evaluations
+        end
+        
+      call_nonblock:
+        test_proc = concurrent_proc{}
+        batch = Array.new(100)
+        
+        while elapsed_seconds < 1
+          batch.each{ test_proc.call_nonblock }
+        end
+        
+      call_detached:
+        test_proc = concurrent_proc{}
+        batch = Array.new(100)
+        
+        while elapsed_seconds < 1
+          evaluations = batch.map{ test_proc.call_detached }
+          evaluations.each{ |evaluation| evaluation.await_result }
+        end
+        
+      call_and_forget:
+        test_proc = concurrent_proc{}
+        batch = Array.new(100)
+        
+        while elapsed_seconds < 1
+          batch.each{ test_proc.call_and_forget }
+          wait 0
+        end
+        
+      waiting call:
+        test_proc = concurrent_proc{ wait 0 }
+        batch = Array.new(100)
+        
+        while elapsed_seconds < 1
+          batch.each{ test_proc.call }
+          # Concurrently::Proc#call already synchronizes the results of evaluations
+        end
+        
+      waiting call_nonblock:
+        test_proc = concurrent_proc{ wait 0 }
+        batch = Array.new(100)
+        
+        while elapsed_seconds < 1
+          evaluations = batch.map{ test_proc.call_nonblock }
+          evaluations.each{ |evaluation| evaluation.await_result }
+        end
+        
+      waiting call_detached:
+        test_proc = concurrent_proc{ wait 0 }
+        batch = Array.new(100)
+        
+        while elapsed_seconds < 1
+          evaluations = batch.map{ test_proc.call_detached }
+          evaluations.each{ |evaluation| evaluation.await_result }
+        end
+        
+      waiting call_and_forget:
+        test_proc = concurrent_proc{ wait 0 }
+        batch = Array.new(100)
+        
+        while elapsed_seconds < 1
+          batch.each{ test_proc.call_and_forget }
+          wait 0
+        end
+        
+    Results for ruby 2.4.1
+    ----------------------
+      call:                       687600 executions in 1.0001 seconds
+      call_nonblock:              855600 executions in 1.0001 seconds
+      call_detached:              426400 executions in 1.0000 seconds
+      call_and_forget:            722200 executions in 1.0000 seconds
+      waiting call:                90300 executions in 1.0005 seconds
+      waiting call_nonblock:      191800 executions in 1.0001 seconds
+      waiting call_detached:      190300 executions in 1.0003 seconds
+      waiting call_and_forget:    207100 executions in 1.0001 seconds
     
-    Results
-    -------
-      # CODE #
-      conproc.call:               76300 executions in 1.0006 seconds
-      conproc.call_nonblock:     186200 executions in 1.0002 seconds
-      conproc.call_detached:     180200 executions in 1.0000 seconds
-      conproc.call_and_forget:   193500 executions in 1.0004 seconds
+    Results for mruby 1.3.0
+    -----------------------
+      call:                       319900 executions in 1.0003 seconds
+      call_nonblock:              431700 executions in 1.0002 seconds
+      call_detached:              158400 executions in 1.0006 seconds
+      call_and_forget:            397700 executions in 1.0002 seconds
+      waiting call:                49900 executions in 1.0015 seconds
+      waiting call_nonblock:       74600 executions in 1.0001 seconds
+      waiting call_detached:       73300 executions in 1.0006 seconds
+      waiting call_and_forget:     85200 executions in 1.0008 seconds
 
+`wait 0` is used as a stand in for all wait methods. Measurements of concurrent
+procs doing nothing are included for comparision.
 
 Explanation of the results:
 
-* `#call` does not profit from batching due to is synchronizing nature.
-* The other methods show an increased throughput compared to running just a
-  single evaluation per event loop iteration.
+* [Concurrently::Proc#call][] is the slowest if the concurrent proc needs to
+  wait. Immediately synchronizing the result for each and every evaluation
+  introduces a noticeable overhead.
+* [Concurrently::Proc#call_nonblock][] and [Concurrently::Proc#call_detached][]
+  perform similarly. When started [Concurrently::Proc#call_nonblock][] skips
+  some work related to waiting that [Concurrently::Proc#call_detached][] is
+  already doing. Now, when the concurrent proc actually waits
+  [Concurrently::Proc#call_nonblock][] needs to make up for this skipped work.
+  This puts its performance in the same region as the one of
+  [Concurrently::Proc#call_detached][].
+* [Concurrently::Proc#call_and_forget][] is the fastest way to wait inside a
+  concurrent proc. It comes at the cost that the result of the evaluation
+  cannot be returned.
+
+To find the fastest way to evaluate a proc it has to be considered if the proc
+does or does not wait most of the time and if its result is needed:
+
+<table>
+  <tr>
+    <th></th>
+    <th>result needed</th>
+    <th>result not needed</th>
+  </tr>
+  <tr>
+    <th>waits almost always</th>
+    <td><code>#call_nonblock</code> or<br/><code>#call_detached</code></td>
+    <td><code>#call_and_forget</code></td>
+  </tr>
+  <tr>
+    <th>waits almost never</th>
+    <td><code>#call_nonblock</code></td>
+    <td><code>#call_nonblock</code></td>
+  </tr>
+</table>
 
-The result of this benchmark is the upper bound for how many concurrent
-evaluations Concurrently is able to run per second. The number of executions
-does not change much with a varying batch size. Larger batches (e.g. 200+)
-gradually start to get a bit slower. A batch of 1000 evaluations still handles
-around 140k executions.
+[Kernel#concurrently][] calls [Concurrently::Proc#call_detached][] under the
+hood as a reasonable default. [Concurrently::Proc#call_detached][] has the
+easiest interface and provides good performance especially in the most common
+use case of Concurrently: waiting for an event to happen.
+[Concurrently::Proc#call_nonblock][] and [Concurrently::Proc#call_and_forget][]
+are there to squeeze out more performance in some edge cases.
 
-You can run the benchmark yourself by running the [script][perf/concurrent_proc_calls_awaiting.rb]:
+You can run this benchmark yourself by executing:
 
-    $ perf/concurrent_proc_calls_awaiting.rb 100
+    $ rake benchmark[call_methods_waiting]
 
 
-[perf/concurrent_proc_calls.rb]: https://github.com/christopheraue/m-ruby-concurrently/blob/master/perf/concurrent_proc_calls.rb
-[perf/concurrent_proc_calls_awaiting.rb]: https://github.com/christopheraue/m-ruby-concurrently/blob/master/perf/concurrent_proc_calls_awaiting.rb
-[Troubleshooting/A_concurrent_proc_is_scheduled_but_never_run]: http://www.rubydoc.info/github/christopheraue/m-ruby-concurrently/file/guides/Troubleshooting.md#A_concurrent_proc_is_scheduled_but_never_run
+[Troubleshooting/A_concurrent_proc_is_scheduled_but_never_run]: http://www.rubydoc.info/github/christopheraue/m-ruby-concurrently/file/guides/Troubleshooting.md#A_concurrent_proc_is_scheduled_but_never_run
+[Concurrently::Proc]: http://www.rubydoc.info/github/christopheraue/m-ruby-concurrently/Concurrently/Proc
+[Concurrently::Proc#call]: http://www.rubydoc.info/github/christopheraue/m-ruby-concurrently/Concurrently/Proc#call-instance_method
+[Concurrently::Proc#call_nonblock]: http://www.rubydoc.info/github/christopheraue/m-ruby-concurrently/Concurrently/Proc#call_nonblock-instance_method
+[Concurrently::Proc#call_detached]: http://www.rubydoc.info/github/christopheraue/m-ruby-concurrently/Concurrently/Proc#call_detached-instance_method
+[Concurrently::Proc#call_and_forget]: http://www.rubydoc.info/github/christopheraue/m-ruby-concurrently/Concurrently/Proc#call_and_forget-instance_method
+[Kernel#concurrently]: http://www.rubydoc.info/github/christopheraue/m-ruby-concurrently/Kernel#concurrently-instance_method

data/guides/Troubleshooting.md

@@ -3,7 +3,7 @@
 To get an idea about the inner workings of Concurrently have a look at the
 [Flow of control][] section in the overview.
 
-## A concurrent proc is scheduled but never run
+## An evaluation is scheduled but never run
 
 Consider the following script:
 
@@ -23,11 +23,11 @@ Running it will only print:
 Unicorns!
 ```
 
-`concurrently{}` is a shortcut for `concurrent_proc{}.call_and_forget`
+`concurrently{}` is a shortcut for `concurrent_proc{}.call_detached`
 which in turn does not evaluate its code right away but schedules it to run
-during the next iteration of the event loop. But, since the root evaluation did
-not await anything the event loop has never been entered and the evaluation of
-the concurrent proc has never been started.
+during the next iteration of the event loop. But, since the main evaluation did
+not await anything the event loop has never been entered and the concurrent
+evaluation has never been started.
 
 A more subtle variation of this behavior occurs in the following scenario:
 
@@ -49,10 +49,10 @@ Running it will also only print:
 Unicorns!
 ```
 
-This time, the root evaluation does await something, namely the end of a one
+This time, the main evaluation does await something, namely the end of a one
 second time frame. Because of this, the evaluation of the `concurrently` block
 is indeed started and immediately waits for two seconds. After one second the
-root evaluation is resumed and exits. The `concurrently` block is never awoken
+main evaluation is resumed and exits. The `concurrently` block is never awoken
 again from its now eternal beauty sleep.
 
 ## A call is blocking the entire execution.
@@ -83,8 +83,8 @@ r.await_readable
 r.readpartial 32
 ```
 
-we suspend the root evaluation, switch to the event loop which runs the
-`concurrently` block and once there is something to read from `r` the root
+we suspend the main evaluation, switch to the event loop which runs the
+`concurrently` block and once there is something to read from `r` the main
 evaluation is resumed.
 
 This approach is not perfect. It is not very efficient if we do not need to
@@ -105,28 +105,28 @@ end
 
 ## The event loop is jammed by too many or too expensive evaluations
 
-Let's talk about a concurrent proc with an infinite loop:
+Let's talk about a concurrent evaluation with an infinite loop:
 
 ```ruby
-evaluation = concurrent_proc do
+evaluation = concurrently do
   loop do
     puts "To infinity! And beyond!"
   end
-end.call_detached
+end
 
 concurrently do
   evaluation.conclude_to :cancelled
 end
 ```
 
-When the concurrent proc is scheduled to run it runs and runs and runs and
-never finishes. The event loop is never entered again and the other concurrent
-proc concluding the evaluation is never started.
+When the loop evaluation is scheduled to run it runs and runs and runs and
+never finishes. The event loop is never entered again and the other evaluation
+concluding the evaluation is never started.
 
 A less extreme example is something like:
 
 ```ruby
-concurrent_proc do
+concurrently do
   loop do
     wait 0.1
     puts "timer triggered at: #{Time.now.strftime('%H:%M:%S.%L')}"
@@ -134,7 +134,7 @@ concurrent_proc do
       sleep 1 # defers the entire event loop
     end
   end
-end.call
+end.await_result
 
 # => timer triggered at: 16:08:17.704
 # => timer triggered at: 16:08:18.705
@@ -175,20 +175,20 @@ managing IOs (e.g. closing them).
 
 ## Errors tear down the event loop
   
-Every concurrent proc rescues the following errors happening during its
-evaluation: `NoMemoryError`, `ScriptError`, `SecurityError`, `StandardError`
-and `SystemStackError`. These are all errors that should not have an immediate
-influence on other evaluations or the application as a whole. They will not
-leak to the event loop and will not tear it down.
+Every evaluation rescues the following errors: `NoMemoryError`, `ScriptError`,
+`SecurityError`, `StandardError` and `SystemStackError`. These are all errors
+that should not have an immediate influence on other evaluations or the
+application as a whole. They will not leak to the event loop and will not tear
+it down.
 
-All other errors happening inside a concurrent proc *will* tear down the
-event loop. These error types are: `SignalException`, `SystemExit` and the
-general `Exception`. In such a case the event loop exits by raising a
-[Concurrently::Error][].
+All other errors happening during an evaluation *will* tear down the event
+loop. These error types are: `SignalException`, `SystemExit` and the general
+`Exception`. In such a case the event loop exits by re-raising the causing
+error.
 
 If your application rescues the error when the event loop is teared down
 and continues running (irb does this, for example) it will do so with a
-[reinitialized event loop] [Concurrently::EventLoop#reinitialize!].
+[reinitialized event loop][Concurrently::EventLoop#reinitialize!].
 
 ## Using Plain Fibers
 
@@ -196,7 +196,7 @@ In principle, you can safely use plain ruby fibers alongside concurrent procs.
 Just make sure you are exclusively operating on these fibers to not
 accidentally interfere with the fibers managed by Concurrently. Be
 especially careful with `Fiber.yield` and `Fiber.current` inside a concurrent
-proc.
+evaluation.
 
 ## Fiber-local variables are treated as thread-local