polyphony 0.43.8

data/docs/main-concepts/concurrency.md

@@ -0,0 +1,151 @@
+---
+layout: page
+title: Concurrency the Easy Way
+nav_order: 1
+parent: Main Concepts
+permalink: /main-concepts/concurrency/
+prev_title: Tutorial
+next_title: How Fibers are Scheduled
+---
+# Concurrency the Easy Way
+
+Concurrency is a major consideration for modern programmers. Applications and
+digital platforms are nowadays expected to do multiple things at once: serve
+multiple clients, process multiple background jobs, talk to multiple external
+services. Concurrency is the property of our programming environment allowing us
+to schedule and control multiple ongoing operations.
+
+Traditionally, concurrency has been achieved by using multiple processes or
+threads. Both approaches have proven problematic. Processes consume relatively a
+lot of memory, and are relatively difficult to coordinate. Threads consume less
+memory than processes and make it difficult to synchronize access to shared
+resources, often leading to race conditions and memory corruption. Using threads
+often necessitates either using special-purpose thread-safe data structures, or
+otherwise protecting shared resource access using mutexes and critical sections.
+In addition, dynamic languages such as Ruby and Python will synchronize multiple
+threads using a global interpreter lock, which means thread execution cannot be
+parallelized. Furthermore, the amount of threads and processes on a single
+system is relatively limited, to the order of several hundreds or a few thousand
+at most.
+
+Polyphony offers a third way to write concurrent programs, by using a Ruby
+construct called [fibers](https://ruby-doc.org/core-2.6.5/Fiber.html). Fibers,
+based on the idea of [coroutines](https://en.wikipedia.org/wiki/Coroutine),
+provide a way to run a computation that can be suspended and resumed at any
+moment. For example, a computation waiting for a reply from a database can
+suspend itself, transferring control to another ongoing computation, and be
+resumed once the database has sent back its reply. Meanwhile, another
+computation is started that opens a socket to a remote service, and then
+suspends itself, waiting for the connection to be established.
+
+This form of concurrency, called cooperative concurrency (in contrast to
+pre-emptive concurrency, like in threads and processes), offers many advantages,
+especially for applications that are [I/O
+bound](https://en.wikipedia.org/wiki/I/O_bound). Fibers are very lightweight
+(starting at about 10KB), can be context-switched faster than threads or
+processes, and literally millions of them can be created on a single system -
+the only limiting factor is available memory.
+
+Polyphony takes Ruby's fibers and adds a way to schedule and switch between them
+automatically whenever a blocking operation is started, such as waiting for a
+TCP connection to be established, for incoming data on an HTTP conection, or for
+a timer to elapse. In addition, Polyphony patches the stock Ruby classes to
+support its concurrency model, letting developers use all of Ruby's stdlib, for
+example `Net::HTTP` and `Mail` while reaping the benefits of lightweight,
+fine-grained, performant, fiber-based concurrency.
+
+Writing concurrent applications using Polyphony's fiber-based concurrency model
+offers a significant performance advantage. Complex concurrent tasks can be
+broken down into many fine-grained concurrent operations with very low overhead.
+More importantly, this concurrency model lets developers express their ideas in
+a sequential fashion, leading to source code that is much easier to read and
+understand, compared to callback-style programming.
+
+## Fibers - Polyphony's basic unit of concurrency
+
+Polyphony extends the core `Fiber` class with additional functionality that
+allows scheduling, synchronizing, interrupting and otherwise controlling running
+fibers. Starting a concurrent operation inside a fiber is as simple as a `spin`
+method call:
+
+```ruby
+while (connection = server.accept)
+  spin { handle_connection(connection) }
+end
+```
+
+In order to facilitate developing applications that employ complex concurrent
+patterns and can scale easily, Polyphony employs a [structured
+approach](https://en.wikipedia.org/wiki/Structured_concurrency) to controlling
+fiber lifetime. A spun fiber is considered the *child* of the fiber from which
+it was spun, and is always limited to the life time of its parent:
+
+```ruby
+parent = spin do
+  do_something
+  child = spin do
+    do_some_other_stuff
+  end
+  # the child fiber is guaranteed to stop executing before the parent fiber
+  # terminates
+end
+```
+
+Any uncaught exception raised in a fiber will be
+[propagated]((exception-handling.md)) to its parent, and potentially further up
+the fiber hierarchy, all the way to the main fiber:
+
+```ruby
+parent = spin do
+  child = spin do
+    raise 'foo'
+  end
+  sleep
+end
+
+sleep
+# the exception will be propagated from the child fiber to the parent fiber,
+# and from the parent fiber to the main fiber, which will cause the program to
+# abort.
+```
+
+In addition, fibers can communicate with each other using message passing,
+turning them into autonomous actors in a highly concurrent environment. Message
+passing is in many ways a superior way to pass data between concurrent entities,
+obviating the need to synchronize access to shared resources:
+
+```ruby
+writer = spin do
+  while (write_request = receive)
+    do_write(write_request)
+  end
+end
+...
+writer << { stamp: Time.now, value: rand }
+```
+
+## Higher-Order Concurrency Constructs
+
+Polyphony also provides several methods and constructs for controlling multiple
+fibers. Methods like `cancel_after` and `move_on_after` allow interrupting a
+fiber that's blocking on any arbitrary operation.
+
+Some other constructs offered by Polyphony:
+
+* `Mutex` - a mutex used to synchronize access to a single shared resource.
+* `ResourcePool` - used for synchronizing access to a limited amount of shared 
+  resources, for example a pool of database connections.
+* `Throttler` - used for throttling repeating operations, for example throttling
+  access to a shared resource, or throttling incoming requests.
+
+## A Compelling Concurrency Solution for Ruby
+
+> The goal of Ruby is to make programmers happy.
+
+— Yukihiro “Matz” Matsumoto
+
+Polyphony's goal is to make programmers even happier by offering them an easy
+way to write concurrent applications in Ruby. Polyphony aims to show that Ruby
+can be used for developing sufficiently high-performance applications, while
+offering all the advantages of Ruby, with source code that is easy to read and
+understand.

data/docs/main-concepts/design-principles.md

@@ -0,0 +1,161 @@
+---
+layout: page
+title: The Design of Polyphony
+nav_order: 5
+parent: Main Concepts
+permalink: /main-concepts/design-principles/
+prev_title: Extending Polyphony
+---
+# The Design of Polyphony
+
+Polyphony is a new gem that aims to enable developing high-performance
+concurrent applications in Ruby using a fluent, compact syntax and API.
+Polyphony enables fine-grained concurrency - the splitting up of operations into
+a large number of concurrent tasks, each concerned with small part of the whole
+and advancing at its own pace. Polyphony aims to solve some of the problems
+associated with concurrent Ruby programs using a novel design that sets it apart
+from other approaches currently being used in Ruby.
+
+## Origins
+
+The Ruby core language (at least in its MRI implementation) currently provides
+two main constructs for performing concurrent work: threads and fibers. While
+Ruby threads are basically wrappers for OS threads, fibers are essentially
+continuations, allowing pausing and resuming distinct computations. Fibers have
+been traditionally used mostly for implementing enumerators and generators.
+
+In addition to the core Ruby concurrency primitives, some Ruby gems have been
+offering an alternative solution to writing concurrent Ruby apps, most notably
+[EventMachine](https://github.com/eventmachine/eventmachine/), which implements
+an event reactor and offers an asynchronous callback-based API for writing
+concurrent code.
+
+In the last couple of years, however, fibers have been receiving more attention
+as a possible constructs for writing concurrent programs. In particular, the
+[Async](https://github.com/socketry/async) framework, created by [Samuel
+Williams](https://github.com/ioquatix), offering a comprehensive set of
+libraries, employs fibers in conjunction with an event reactor provided by the
+[nio4r](https://github.com/socketry/nio4r) gem, which wraps the C
+library [libev](http://software.schmorp.de/pkg/libev.html).
+
+In addition, recently some effort was undertaken to provide a way to
+[automatically switch between fibers](https://bugs.ruby-lang.org/issues/13618)
+whenever a blocking operation is performed, or to [integrate a fiber
+scheduler](https://bugs.ruby-lang.org/issues/16786) into the core Ruby code.
+
+Nevertheless, while work is being done to harness fibers for providing a better
+way to do concurrency in Ruby, fibers remain a mistery for most Ruby
+programmers, a perplexing unfamiliar corner right at the heart of Ruby.
+
+## The History of Polyphony
+
+Polyphony started as an experiment, but over about two years of slow, jerky
+evolution turned into something I'm really excited to share with the Ruby
+community. Polyphony's design is both similar and different than the projects
+mentioned above.
+
+Polyphony today as nothing like the way it began. A careful examination of the
+[CHANGELOG](https://github.com/digital-fabric/polyphony/blob/master/CHANGELOG.md)
+would show how Polyphony explored not only different event reactor designs, but
+also different API designs incorporating various concurrent paradigms such as
+promises, async/await, fibers, and finally structured concurrency.
+
+Throughout the development process, it was my intention to create a programming
+interface that would make it easy to author highly-concurrent Ruby programs.
+
+## Design Principles
+
+While Polyphony, like nio4r or EventMachine, uses an event reactor to turn
+blocking operations into non-blocking ones, it completely embraces fibers and in
+fact does not provide any callback-based APIs.
+
+Furthermore, Polyphony provides fullblown fiber-aware implementations of
+blocking operations, such as `read/write`, `sleep` or `waitpid`, instead of just
+event watching primitives.
+
+Polyphony's design is based on the following principles:
+
+- The concurrency model should feel "baked-in". The API should allow
+  concurrency with minimal effort. Polyphony should facilitate writing both
+  large apps and small scripts with as little boilerplate code as possible.
+  There should be no calls to initialize the event reactor, or other ceremonial
+  code:
+
+  ```ruby
+  require 'polyphony'
+
+  # start 10 fibers, each sleeping for 3 seconds
+  10.times { spin { sleep 3 } }
+
+  puts 'going to sleep now'
+  # wait for other fibers to terminate
+  suspend
+  ```
+
+- Blocking operations should yield to other concurrent tasks without any
+  decoration or wrapper APIs. This means no `async/await` notation, and no
+  async callback-style APIs.
+
+  ```ruby
+  # in Polyphony, I/O ops might block the current fiber, but implicitly yield to
+  # other concurrent fibers:
+  clients.each do |client|
+    spin { client.puts 'Elvis has left the chatroom' }
+  end
+  ```
+
+- Concurrency primitives should be accessible using idiomatic Ruby techniques
+  (blocks, method chaining...) and should feel as much as possible "part of the
+  language". The resulting API is fundamentally based on methods rather than classes,
+  for example `spin` or `move_on_after`, leading to a coding style that is both
+  more compact and more legible:
+
+  ```ruby
+  fiber = spin {
+    move_on_after(3) {
+      do_something_slow
+    }
+  }
+  ```
+
+- Polyphony should embrace Ruby's standard `raise/rescue/ensure` exception
+  handling mechanism. Exception handling in a highly concurrent environment
+  should be robust and foolproof:
+
+  ```ruby
+  cancel_after(0.5) do
+    puts 'going to sleep'
+    sleep 1
+    # this will not be printed
+    puts 'wokeup'
+  ensure
+    # this will be printed
+    puts 'done sleeping'
+  end
+  ```
+
+- Concurrency primitives should allow creating higher-order concurrent
+  constructs through composition.
+
+- The entire design should embrace fibers. There should be no callback-based
+  asynchronous APIs. The library and its ecosystem will foster the development
+  of techniques and tools for converting callback-based APIs to fiber-based ones.
+
+- Use of extensive monkey patching of Ruby core modules and classes such as
+  `Kernel`, `Fiber`, `IO` and `Timeout`. This allows porting over non-Polyphony
+  code, as well as using a larger part of stdlib in a concurrent manner, without
+  having to use custom non-standard network classes or other glue code.
+
+  ```ruby
+  require 'polyphony'
+
+  # use TCPServer from Ruby's stdlib
+  server = TCPServer.open('127.0.0.1', 1234)
+  while (client = server.accept)
+    spin {
+      while (data = client.gets)
+        client.write("you said: #{ data.chomp }\n")
+      end
+    }
+  end
+  ```

data/docs/main-concepts/exception-handling.md

@@ -0,0 +1,291 @@
+---
+layout: page
+title: Exception Handling
+nav_order: 3
+parent: Main Concepts
+permalink: /main-concepts/exception-handling/
+prev_title: How Fibers are Scheduled
+next_title: Extending Polyphony
+---
+# Exception Handling
+
+Ruby employs a pretty robust exception handling mechanism. An raised exception
+will propagate up the fiber tree until a suitable exception handler is found,
+based on the exception's class. In addition, the exception will include a stack
+trace showing the execution path from the exception's locus back to the
+program's entry point. Unfortunately, when exceptions are raised while switching
+between fibers, stack traces will only include partial information. Here's a
+simple demonstration:
+
+_fiber\_exception.rb_
+
+```ruby
+require 'fiber'
+
+def fail!
+  raise 'foobar'
+end
+
+f = Fiber.new do
+  Fiber.new do
+    fail!
+  end.transfer
+end
+
+f.transfer
+```
+
+Running the above program will give us:
+
+```text
+Traceback (most recent call last):
+        1: from fiber_exception.rb:9:in `block (2 levels) in <main>'
+fiber_exception.rb:4:in `fail!': foobar (RuntimeError)
+```
+
+So, the stack trace includes two frames: the exception's locus on line 4 and the
+call site at line 9. But we have no information on how we got to line 9. Let's
+imagine if we had more complete information about the sequence of execution. In
+fact, what is missing is information about how the different fibers were
+created. If we had that, our stack trace would have looked something like this:
+
+```text
+Traceback (most recent call last):
+        4: from fiber_exception.rb:13:in `<main>'
+        3: from fiber_exception.rb:7:in `Fiber.new'
+        2: from fiber_exception.rb:8:in `Fiber.new'
+        1: from fiber_exception.rb:9:in `block (2 levels) in <main>'
+fiber_exception.rb:4:in `fail!': foobar (RuntimeError)
+```
+
+In order to achieve this, Polyphony patches `Fiber.new` to keep track of the
+call stack at the moment the fiber was created, as well as the fiber from which
+the call happened. In addition, Polyphony patches `Exception#backtrace` in order
+to synthesize a complete stack trace based on the call stack information stored
+for the current fiber. This is done recursively through the chain of fibers
+leading up to the current location. What we end up with is a record of the
+entire sequence of \(possibly intermittent\) execution leading up to the point
+where the exception was raised.
+
+In addition, the backtrace is sanitized to remove stack frames originating from
+the Polyphony code itself, which hides away the Polyphony plumbing and lets
+developers concentrate on their own code. The sanitizing of exception backtraces
+can be disabled by setting the `Exception.__disable_sanitized_backtrace__` flag:
+
+```ruby
+Exception.__disable_sanitized_backtrace__ = true
+...
+```
+
+## Exceptions and Fiber Scheduling
+
+Polyphony takes advantages of Ruby's `Fiber#transfer` API to allow interrupting
+fiber execution and raise cross-fiber exceptions. This is done by inspecting the
+return value of `Fiber#transfer`, which returns when the fiber resumes, at every
+[switchpoint](../fiber-scheduling/#switchpoints). If the return value is an
+exception, it is raised in the context of the resumed fiber, and is then subject
+to any `rescue` statements in the context of that fiber.
+
+Exceptions can be passed to arbitrary fibers by using `Fiber#raise`. They can also be manually raised in fibers by using `Fiber#schedule`:
+
+```ruby
+f = spin do
+  suspend
+rescue => e
+  puts e.message
+end
+
+f.schedule(RuntimeError.new('foo')) #=> will print 'foo'
+```
+
+## Cleaning Up After Exceptions - Using Ensure
+
+A major issue when handling exceptions is cleaning up - freeing up resources
+that have been allocated, cancelling ongoing operations, etc. Polyphony allows
+using the normal `ensure` statement for cleaning up. Have a look at Polyphony's
+implementation of `Kernel#sleep`:
+
+```ruby
+def sleep(duration)
+  timer = Gyro::Timer.new(duration, 0)
+  timer.await
+ensure
+  timer.stop
+end
+```
+
+This method creates a one-shot timer with the given duration and then suspends
+the current fiber, waiting for the timer to fire and then resume the fiber.
+While the awaiting fiber is suspended, other operations might be going on, which
+might interrupt the `sleep` operation by scheduling the awaiting fiber with an
+exception, for example a `MoveOn` or a `Cancel` exception. For this reason, we
+need to _ensure_ that the timer will be stopped, regardless of whether it has
+fired or not. We call `timer.stop` inside an ensure block, thus ensuring that
+the timer will have stopped once the awaiting fiber has resumed, even if it has
+not fired.
+
+## Exception Propagation
+
+One of the "annoying" things about exceptions is that for them to be useful, you
+have to intercept them \(using `rescue`\). If you forget to do that, you'll end
+up with uncaught exceptions that can wreak havoc. For example, by default a Ruby
+`Thread` in which an exception was raised without being caught, will simply
+terminate with the exception silently swallowed.
+
+To prevent the same from happening with fibers, Polyphony provides a robust
+mechanism that propagates uncaught exceptions up through the chain of parent
+fibers. Let's discuss the following example:
+
+```ruby
+require 'polyphony'
+
+spin do
+  spin do
+    spin do
+      spin do
+        raise 'foo'
+      end
+      sleep
+    end
+    sleep
+  end
+  sleep
+end
+
+sleep
+```
+
+In the above example, four nested fibers are created, and each of them, except
+for the innermost fiber, goes to sleep for an unlimited duration. An exception
+is raised in the innermost fiber, and having no corresponding exception handler,
+will propagate up through the enclosing fibers, until reaching the
+top-most level, that of the root fiber, at which point the exception will cause
+the program to abort and print an error message.
+
+## MoveOn and Cancel - Interrupting Fiber Execution
+
+In addition to enhancing Ruby's normal exception-handling mechanism, Polyphony
+provides two exception classes that used exclusively to interrupt fiber
+execution: `MoveOn` and `Cancel`. Both of these classes are used in various
+fiber-control APIs, and `MoveOn` exceptions in particular are handled in a
+particular manner by Polyphony. The difference between `MoveOn` and `Cancel` is
+that `MoveOn` stops fiber execution without the exception propagating. It can
+optionally provide an arbitrary return value for the fiber. `Cancel` will propagate
+up like all exceptions.
+
+The `MoveOn` and `Cancel` classes are normally used indirectly, through the
+`Fiber#interrupt` and `Fiber#cancel` APIs, and also through the use of [cancel
+scopes](#):
+
+```ruby
+f1 = spin { sleep 100; return 'foo' }
+f2 = spin { f1.await }
+...
+f1.interrupt('bar')
+f2.result #=> 'bar'
+
+f3 = spin { sleep 100 }
+...
+f3.cancel #=> will raise a Cancel exception
+```
+
+In addition to `MoveOn` and `Cancel`, Polyphony employs internally another
+exception class, `Terminate` for terminating a fiber once its parent has
+finished executing.
+
+## The Special Problem of Signal Handling
+
+Ruby by default handles process signals by generating exceptions, allowing the
+handling of signals in a structured manner. However, process signals may arrive
+at any moment, and may be trapped while any arbitrary fiber is running, and even
+while an event loop is running.
+
+Two signals in particular require special care as they involve the stopping of
+the entire process: `TERM` and `INT`. The `TERM` signal should be handled
+gracefully, i.e. with proper cleanup, which also means terminating all fibers.
+The `INT` signal requires halting the process and printing a correct stack
+trace.
+
+To ensure correct behaviour for these two signals, polyphony installs signal
+handlers that ensure that the main thread's event loop stops if it's currently
+running, and that the corresponding exceptions (namely `SystemExit` and
+`Interrupt`) are handled correctly by passing them to the main fiber.
+
+### Graceful process termination
+
+In order to ensure your application terminates gracefully upon receiving an
+`INT` or `TERM` signal, you'll need to:
+
+1. Rescue the corresponding exceptions in the main fiber.
+2. Rescue `Polyphony::Terminate` exceptions in each fiber that needs to perform
+   operations such as handling any pending requests, etc.
+
+```ruby
+# In a worker fiber
+def do_work
+  loop do
+    req = receive
+    handle_req(req)
+  end
+rescue Polyphony::Terminate
+  # We still need to handle any pending request
+  receive_pending.each { handle_req(req) }
+end
+
+# on the main fiber
+begin
+  spin_up_lots_fibers
+rescue Interrupt, SystemExit
+  Fiber.current.terminate_all_children
+  Fiber.current.await_all_children
+end
+```
+
+### Handling other signals
+
+Care should be taken when handling other signals. There are two options for
+correctly handling the signals: using Ruby's stock `trap` method, and using
+Polyphony's signal watchers. The stock method involves trapping signals as
+usual, but making sure we're not inside the event loop:
+
+```ruby
+trap('SIGHUP') do
+  Thread.current.break_out_of_ev_loop(Thread.current.main_fiber, nil)
+  handle_hup_signal
+end
+```
+
+A second technique that might be useful is to use a `Gyro::Async` watcher and
+signal it when the process signal is trapped:
+
+```ruby
+sighup_async = Gyro::Async.new
+sighup_handler = spin_loop do
+  sighup_async.await
+  handle_sighup
+end
+
+trap('SIGHUP') { sighup_async.signal }
+```
+
+Another alternative is to use `Polyphony.wait_for_signal`, which uses a
+`Gyro::Signal` watcher under the hood:
+
+```ruby
+hup_handler = spin_loop do
+  Polyphony.wait_for_signal('SIGHUP')
+  handle_hup_signal
+end
+```
+
+## The Special Problem of Thread Termination
+
+Thread termination using `Thread#kill` or `Thread#raise` also presents the same
+problems as signal handling in a multi-fiber environment. The termination can
+occur while any fiber is running, and even while running the thread's event
+loop.
+
+To ensure proper thread termination, including the termination of all the
+thread's fibers, Polyphony patches the `Thread#kill` and `Thread#raise` methods
+to schedule the thread's main fiber with the corresponding exceptions, thus
+ensuring an orderly termination or exception handling.