polyphony 0.36 → 0.42

data/docs/getting-started/tutorial.md

@@ -1,13 +1,20 @@
 ---
 layout: page
 title: Tutorial
-nav_order: 2
 parent: Getting Started
-permalink: /getting-started/tutorial/
-prev_title: Installing Polyphony
-next_title: Concurrency the Easy Way
+nav_order: 3
+---
+
+# Tutorial
+{: .no_toc }
+
+## Table of contents
+{: .no_toc .text-delta }
+
+- TOC
+{:toc}
+
 ---
-# Polyphony: a Tutorial
 
 Polyphony is a new Ruby library aimed at making writing concurrent Ruby apps
 easy and fun. In this article, we'll introduce Polyphony's fiber-based
@@ -116,7 +123,7 @@ suspend # The main fiber suspends, waiting for all other work to finish
   sleep 1 # The sleeper fiber goes to sleep
     Gyro::Timer.new(1, 0).await # A timer event watcher is setup and yields
       Thread.current.switch_fiber # Polyphony looks for other runnable fibers
-        Thread.current.selector.run # With no work left, the event loop is ran
+        Thread.current.agent.poll # With no work left, the event loop is ran
           fiber.schedule # The timer event fires, scheduling the sleeper fiber
   # <= The sleep method returns
   puts "Woke up"
@@ -218,7 +225,7 @@ this by setting up a timeout fiber that cancels the fiber dealing with the conne
 def handle_client(client)
   timeout = cancel_after(10)
   while (data = client.gets)
-    timeout.reset
+    timeout.restart
     client << data
   end
 rescue Polyphony::Cancel
@@ -230,18 +237,19 @@ ensure
 end
 ```
 
-The cancel scope is initialized with a timeout of 10 seconds. Any blocking
-operation ocurring within the cancel scope will be interrupted once 10 seconds
-have elapsed. In order to keep the connection alive while the client is active,
-we call `scope.reset_timeout` each time data is received from the client, and
-thus reset the cancel scope timer.
-
-In addition, we use an `ensure` block to make sure the client connection is
-closed, whether or not it was interrupted by the cancel scope timer. The habit
-of always cleaning up using `ensure` in the face of potential interruptions is a
-fundamental element of using Polyphony correctly. This makes your code robust,
-even in a highly chaotic concurrent execution environment where tasks can be
-interrupted at any time.
+The call to `#cancel_after` spins up a new fiber that will sleep for 10 seconds,
+then cancel its parent. The call to `client.gets` blocks until new data is
+available. If no new data is available, the `timeout` fiber will finish
+sleeping, and then cancel the client handling fiber by raising a
+`Polyphony::Cancel` exception. However, if new data is received, the `timeout`
+fiber is restarted, causing to begin sleeping again for 10 seconds. If the
+client has closed the connection, or some other exception occurs, the `timeout`
+fiber is automatically stopped as it is a child of the client handling fiber.
+
+The habit of always cleaning up using `ensure` in the face of potential
+interruptions is a fundamental element of using Polyphony correctly. This makes
+your code robust, even in a highly chaotic concurrent execution environment
+where tasks can be started, restarted and interrupted at any time.
 
 ## Implementing graceful shutdown
 

data/docs/index.md

@@ -14,8 +14,9 @@ implements a comprehensive
 using [libev](https://github.com/enki/libev) as a high-performance event reactor
 for I/O, timers, and other asynchronous events.
 
-[FAQ](faq){: .btn .btn-green .text-gamma }
 [Take the tutorial](getting-started/tutorial){: .btn .btn-blue .text-gamma }
+[Main Concepts](main-concepts/concurrency/){: .btn .btn-green .text-gamma }
+[FAQ](faq){: .btn .btn-green .text-gamma }
 [Source code](https://github.com/digital-fabric/polyphony){: .btn .btn-purple .text-gamma target="_blank" }
 {: .mt-6 .h-align-center }
 
@@ -52,7 +53,7 @@ adapters are being developed.
 * Natural, sequential programming style that makes it easy to reason about
   concurrent code.
 * Abstractions and constructs for controlling the execution of concurrent code:
-  supervisors, cancel scopes, throttling, resource pools etc.
+  supervisors, throttling, resource pools etc.
 * Code can use native networking classes and libraries, growing support for
   third-party gems such as `pg` and `redis`.
 * Use stdlib classes such as `TCPServer` and `TCPSocket` and `Net::HTTP`.

data/docs/main-concepts/concurrency.md

@@ -130,16 +130,11 @@ Polyphony also provides several methods and constructs for controlling multiple
 fibers. Methods like `cancel_after` and `move_on_after` allow interrupting a
 fiber that's blocking on any arbitrary operation.
 
-Cancel scopes \(borrowed from the brilliant Python library
-[Trio](https://trio.readthedocs.io/en/stable/)\) allows cancelling ongoing
-operations for any reason with more control over cancelling behaviour.
-
 Some other constructs offered by Polyphony:
 
 * `Mutex` - a mutex used to synchronize access to a single shared resource.
 * `ResourcePool` - used for synchronizing access to a limited amount of shared 
   resources, for example a pool of database connections.
-
 * `Throttler` - used for throttling repeating operations, for example throttling
   access to a shared resource, or throttling incoming requests.
 

data/docs/main-concepts/design-principles.md

@@ -1,18 +1,76 @@
 ---
 layout: page
-title: Design Principles
+title: The Design of Polyphony
 nav_order: 5
 parent: Main Concepts
 permalink: /main-concepts/design-principles/
 prev_title: Extending Polyphony
 ---
-# Design Principles
+# The Design of Polyphony 
+
+Polyphony is a new gem that aims to enable developing high-performance
+concurrent applications in Ruby using a fluent, compact syntax and API.
+Polyphony enables fine-grained concurrency - the splitting up of operations into
+a large number of concurrent tasks, each concerned with small part of the whole
+and advancing at its own pace. Polyphony aims to solve some of the problems
+associated with concurrent Ruby programs using a novel design that sets it apart
+from other approaches currently being used in Ruby.
+
+## Origins
+
+The Ruby core language (at least in its MRI implementation) currently provides
+two main constructs for performing concurrent work: threads and fibers. While
+Ruby threads are basically wrappers for OS threads, fibers are essentially
+continuations, allowing pausing and resuming distinct computations. Fibers have
+been traditionally used mostly for implementing enumerators and generators.
+
+In addition to the core Ruby concurrency primitives, some Ruby gems have been
+offering an alternative solution to writing concurrent Ruby apps, most notably
+[EventMachine](https://github.com/eventmachine/eventmachine/), which implements
+an event reactor and offers an asynchronous callback-based API for writing
+concurrent code.
+
+In the last couple of years, however, fibers have been receiving more attention
+as a possible constructs for writing concurrent programs. In particular, the
+[Async](https://github.com/socketry/async) framework, created by [Samuel
+Williams](https://github.com/ioquatix), offering a comprehensive set of
+libraries, employs fibers in conjunction with an event reactor provided by the
+[nio4r](https://github.com/socketry/nio4r) gem, which wraps the C
+library [libev](http://software.schmorp.de/pkg/libev.html).
+
+In addition, recently some effort was undertaken to provide a way to
+[automatically switch between fibers](https://bugs.ruby-lang.org/issues/13618)
+whenever a blocking operation is performed, or to [integrate a fiber
+scheduler](https://bugs.ruby-lang.org/issues/16786) into the core Ruby code.
+
+Nevertheless, while work is being done to harness fibers for providing a better
+way to do concurrency in Ruby, fibers remain a mistery for most Ruby
+programmers, a perplexing unfamiliar corner right at the heart of Ruby.
+
+## Design Principles
+
+Polyphony started as an experiment, but over about two years of slow, jerky
+evolution turned into something I'm really excited to share with the Ruby
+community. Polyphony's design is both similar and different than the projects
+mentioned above.
+
+Polyphony today as nothing like the way it began. A careful examination of the
+[CHANGELOG](https://github.com/digital-fabric/polyphony/blob/master/CHANGELOG.md)
+would show how Polyphony explored not only different event reactor designs, but
+also different API designs incorporating various concurrent paradigms such as
+promises, async/await, fibers, and finally structured concurrency. 
+
+While Polyphony, like nio4r or EventMachine, uses an event reactor to turn
+blocking operations into non-blocking ones, it completely embraces fibers and in
+fact does not provide any callback-based APIs. Furthermore, Polyphony provides
+fullblown fiber-aware implementations of blocking operations, such as
+`read/write`, `sleep` or `waitpid`, instead of just event watching primitives.
+
+Throughout the development process, it was my intention to create a programming
+interface that would make highly-concurrent 
+
+
 
-Polyphony was created in order to enable developing high-performance concurrent
-applications in Ruby using a fluent, compact syntax and API. Polyphony enables
-fine-grained concurrency - the splitting up of operations into a large number of
-concurrent tasks, each concerned with small part of the whole and advancing at
-its own pace.
 
 
 
@@ -46,8 +104,8 @@ library. Polyphony's design is based on the following principles:
   async callback-style APIs.
 
   ```ruby
-  # in Polyphony, I/O ops block the current fiber, but implicitly yield to other
-  # concurrent fibers:
+  # in Polyphony, I/O ops might block the current fiber, but implicitly yield to
+  # other concurrent fibers:
   clients.each { |client|
     spin { client.puts 'Elvis has left the chatroom' }
   }
@@ -85,18 +143,8 @@ library. Polyphony's design is based on the following principles:
   end
   ```
 
-- Concurrency primitives should allow creating higher-order concurrent 
-  constructs through composition. This is done primarily through supervisors and
-  cancel scopes:
-
-  ```ruby
-  # wait for multiple fibers
-  supervise { |s|
-    clients.each { |client|
-      s.spin { client.puts 'Elvis has left the chatroom' }
-    }
-  }
-  ```
+- Concurrency primitives should allow creating higher-order concurrent
+  constructs through composition.
 
 - The entire design should embrace fibers. There should be no callback-based
   asynchronous APIs.

data/docs/main-concepts/extending.md

@@ -5,7 +5,7 @@ nav_order: 4
 parent: Main Concepts
 permalink: /main-concepts/extending/
 prev_title: Exception Handling
-next_title: Design Principles
+next_title: The Design of Polyphony
 ---
 # Extending Polyphony
 

data/docs/main-concepts/index.md

@@ -0,0 +1,9 @@
+---
+layout: page
+title: Main Concepts
+has_children: true
+nav_order: 3
+---
+
+# Main Concepts
+{: .no_toc }

data/examples/core/01-spinning-up-fibers.rb

@@ -14,4 +14,5 @@ end
 spin { nap(:a, 1) }
 spin { nap(:b, 2) }
 
+# Calling suspend will block until all child fibers have terminated
 suspend

data/examples/core/03-interrupting.rb

@@ -18,7 +18,8 @@ ensure
 end
 
 # The Kernel#cancel_after interrupts a blocking operation by raising a
-# Polyphony::Cancel exception after the given timeout
+# Polyphony::Cancel exception after the given timeout. If not rescued, the
+# exception is propagated up the fiber hierarchy
 spin do
   # cancel after 1 second
   cancel_after(1) { nap(:cancel, 2) }
@@ -26,6 +27,8 @@ rescue Polyphony::Cancel => e
   puts "got exception: #{e}"
 end
 
+# The Kernel#move_on_after interrupts a blocking operation by raising a
+# Polyphony::MoveOn exception, which is silently swallowed by the fiber
 spin do
   # move on after 1 second
   move_on_after(1) do

data/examples/core/04-handling-signals.rb

@@ -0,0 +1,19 @@
+# frozen_string_literal: true
+
+require 'bundler/setup'
+require 'polyphony'
+
+# trap('TERM') do
+#   Polyphony.emit_signal_exception(::SystemExit)
+# end
+
+# trap('INT') do
+#   Polyphony.emit_signal_exception(::Interrupt)
+# end
+
+puts "go to sleep"
+begin
+  sleep
+ensure
+  puts "done sleeping"
+end

data/examples/core/xx-agent.rb

@@ -0,0 +1,102 @@
+# frozen_string_literal: true
+
+require 'bundler/setup'
+require 'polyphony'
+
+Exception.__disable_sanitized_backtrace__ = true
+
+class Test
+  def test_sleep
+    puts "going to sleep"
+    sleep 1
+    puts "done sleeping"
+  end
+  
+  def test_spin
+    spin {
+      10.times {
+        STDOUT << '.'
+        sleep 0.1
+      }
+    }
+    
+    puts "going to sleep\n"
+    sleep 1
+    puts 'woke up'
+  end
+  
+  def test_file
+    f = File.open(__FILE__, 'r')
+    puts Thread.current.agent.read(f, +'', 10000, true)
+    
+    Thread.current.agent.write(STDOUT, "Write something: ")
+    str = +''
+    Thread.current.agent.read(STDIN, str, 5, false)
+    puts str
+  end
+  
+  def test_fork
+    pid = fork do
+      Thread.current.agent.post_fork
+      puts 'child going to sleep'
+      sleep 1
+      puts 'child done sleeping'
+      exit(42)
+    end
+    
+    puts "Waiting for pid #{pid}"
+    result = Thread.current.agent.waitpid(pid)
+    puts "Done waiting"
+    p result
+  end
+  
+  def test_async
+    async = Polyphony::Event.new
+    
+    spin {
+      puts "signaller starting"
+      sleep 1
+      puts "signal"
+      async.signal(:foo)
+    }
+    
+    puts "awaiting event"
+    p async.await
+  end
+  
+  def test_queue
+    q = Gyro::Queue.new
+    spin {
+      10.times {
+        q << Time.now.to_f
+        sleep 0.2
+      }
+      q << :STOP
+    }
+    
+    loop do
+      value = q.shift
+      break if value == :STOP
+      
+      p value
+    end
+  end
+  
+  def test_thread
+    t = Thread.new do
+      puts "thread going to sleep"
+      sleep 0.2
+      puts "thread done sleeping"
+    end
+    
+    t.await
+  end
+end
+
+t = Test.new
+
+t.methods.select { |m| m =~ /^test_/ }.each do |m|
+  puts '*' * 40
+  puts m
+  t.send(m)
+end

data/examples/core/xx-fork-cleanup.rb

@@ -0,0 +1,22 @@
+# frozen_string_literal: true
+
+require 'bundler/setup'
+require 'polyphony'
+
+Exception.__disable_sanitized_backtrace__ = true
+
+t = Thread.new do
+  async = Gyro::Async.new
+  spin { async.await }
+  sleep 100
+end
+
+sleep 0.5
+
+Polyphony.fork do
+  puts "forked #{Process.pid}"
+  sleep 1
+  puts "done sleeping"
+end
+
+sleep 50

data/examples/core/xx-sleeping.rb

@@ -5,13 +5,21 @@ require 'polyphony'
 
 Exception.__disable_sanitized_backtrace__ = true
 
-spin {
-  10.times {
-    STDOUT << '.'
-    sleep 0.1
-  }
-}
+# spin {
+#   10.times {
+#     STDOUT << '.'
+#     sleep 0.1
+#   }
+# }
 
 puts 'going to sleep...'
 sleep 1
 puts 'woke up'
+
+counter = 0
+t = Polyphony::Throttler.new(5)
+t.process do
+  p counter
+  counter += 1
+  t.stop if counter > 5
+end