polyphony 0.28 → 0.29

data/docs/technical-overview/fiber-scheduling.md

@@ -4,19 +4,72 @@ title: How Fibers are Scheduled
 nav_order: 3
 parent: Technical Overview
 permalink: /technical-overview/fiber-scheduling/
+prev_title: Concurrency the Easy Way
+next_title: Exception Handling
 ---
+
 # How Fibers are Scheduled
 
+Before we discuss how fibers are scheduled in Polyphony, let's examine how
+switching between fibers works in Ruby.
+
 Ruby provides two mechanisms for transferring control between fibers:
 `Fiber#resume` /`Fiber.yield` and `Fiber#transfer`. The first is inherently
 asymmetric and is famously used for implementing generators and [resumable
 enumerators](https://blog.appsignal.com/2018/11/27/ruby-magic-fibers-and-enumerators-in-ruby.html).
-Another limiting factor of using resume / yield is that the main fiber can't
-yield away, limiting its usability as a resumable fiber.
+Here's a small example:
+
+```ruby
+fib = Fiber.new do
+  x, y = 0, 1
+  loop do
+    Fiber.yield y
+    x, y = y, x + y
+  end
+end
+
+10.times { puts fib.resume }
+```
+
+Another implication of using resume / yield is that the main fiber can't yield
+away, meaning we cannot pause the main fiber using `Fiber.yield`.
+
+The other fiber control mechanism, using `Fiber#transfer`, is fully symmetric:
+
+```ruby
+require 'fiber'
+
+ping = Fiber.new { loop { puts "ping"; pong.transfer } }
+pong = Fiber.new { loop { puts "pong"; ping.transfer } }
+ping.transfer
+```
+
+`Fiber#transform` also allows using the main fiber as a general purpose
+resumable execution context. Polyphony uses `Fiber#transfer` exclusively for
+scheduling fibers.
+
+## The Different Fiber states
 
-The second mechanism, using `Fiber#transfer`, is completely symmetric and allows
-use of the main fiber as a general purpose resumable execution context.
-Polyphony uses `Fiber#transfer` exclusively for scheduling fibers.
+In Polyphony, each fiber has one four possible states:
+
+A new fiber will start in a `:waiting` state. The `#spin` global method will
+create the fiber and schedule it for execution, marking it as `:runnable`. When
+the fiber is run, it is in a `:running` state. Finally, when the fiber has
+terminated, it transitions to the `:dead` state.
+
+Whenever a fiber performs a blocking operation - such as waiting for a timer to
+elapse, or for a socket to become readable, or for a child process to terminate -
+it transitions to a `:waiting` state. Once the timer has elapsed, the socket
+has become readable, or the child process has terminated, the fiber is marked as
+`:runnable`. It then waits its turn to run.
+
+## Switchpoints
+
+A switchpoint is any point in time at which control might switch from the
+currently running fiber to another fiber that is `:runnable`. This usually
+occurs when the currently running fiber starts a blocking operation. It also
+occurs when the running fiber has yielded control using `#snooze` or `#suspend`.
+A Switchpoint will also occur when the currently running fiber has terminated.
 
 ## Scheduler-less scheduling
 
@@ -27,15 +80,14 @@ reactor-based libraries and frameworks, such as `nio4r`, `EventMachine` or
 user-supplied code *from inside the loop*. In Polyphony, however, we have chosen
 a programming model that does not use a loop to schedule fibers. In fact, in
 Polyphony there's no such thing as a reactor loop, and there's no *scheduler*
-running on a separate execution context.
+per se running on a separate execution context.
 
-Instead, Polyphony maintains for each thread a list of scheduled fibers, fibers
-that will be resumed once the current fiber yields control, which will occur on
-every blocking operation. If no fibers are scheduled, the libev event reactor
-will be ran until one or more events have occurred. Events are handled by adding
-the corresponding fibers onto the *scheduled fibers* list. Finally, control is
-transferred to the first scheduled fiber, which will run until it blocks or
-terminates, at which point control is transferred to the next scheduled fiber.
+Instead, Polyphony maintains for each thread a run queue, a list of `:runnable`
+fibers. If no fiber is `:runnable`, the libev event reactor will be ran until
+one or more events have occurred. Events are handled by adding the corresponding
+fibers onto the run queue. Finally, control is transferred to the first fiber on
+the run queue, which will run until it blocks or terminates, at which point
+control is transferred to the next runnable fiber.
 
 This approach has numerous benefits:
 
@@ -45,89 +97,25 @@ This approach has numerous benefits:
   scheduling code.
 - Much less time is spent in reactor loop callbacks, letting the reactor loop
   run more efficiently.
-- Fibers are resumed outside of the event reactor code, making it easier to
+- Fibers are switched outside of the event reactor code, making it easier to
   avoid race conditions and unexpected behaviours.
 
-## A concrete example - fiber scheduling in an echo server
-
-Here's a barebones echo server written in Polyphony:
-
-```ruby
-require 'polyphony'
-
-server = TCPServer.open('127.0.0.1', 8081)
-while (client = server.accept)
-  spin do
-    while (data = client.gets)
-      client << ">>>you sent: #{data}"
-      break if data =~ /quit/i
-    end
-  end
-end
-```
-
-Let's examine the the flow of control in our echo server program:
-
-<p class="img-figure"><img src="../../assets/img/echo-fibers.svg"></p>
-
-> In the above figure, the fat blue dots represents moments at which fibers can
-> be switched. The light blue horizontal arrows represent switching from one
-> fiber to another. The blue vertical lines represent a train of execution on a
-> single fiber.
-
-- The main fiber (fiber 1) runs a loop waiting for incoming connections.
-- The call to `server.accept` blocks, and an I/O event watcher is set up. The
-  main fiber is suspended.
-- Since there's no other runnable fiber, the associated event loop is run.
-- An event is generated for the server's socket, and fiber 1 is added to the run
-  queue.
-- Fiber 1 is pulled from the run queue and resumed. The `server.accept` call
-  returns a new client socket (an instance of `TCPSocket`).
-- Fiber 1 continues by `spin`ning up a new fiber for handling the client. The
-  new fiber (fiber 2) is added to the run queue.
-- Fiber 1 goes back to waiting for an incoming connection by calling
-  `server.accept` again. The call blocks, the main fiber suspends, and switches
-  to the next fiber in the run queue, fiber 2.
-- Fiber 2 starts a loop and calls `client.gets`. The call blocks, an I/O event
-  watcher is set up, and the fiber suspends.
-- Since no other fiber is runnable, the event loop is run, waiting for at least
-  one event to fire.
-- An event fires for the acceptor socket, and fiber 1 is put on the run queue.
-- An event fires for the client socket, and fiber 2 is put on the run queue.
-- Fiber 1 is resumed and spins up a new client handling fiber (fiber 3), which
-  is put on the run queue.
-- Fiber 1 calls `server.accept` again, and suspends, switching to the next
-  runnable fiber, fiber 2.
-- Fiber 2 resumes and completes reading a line from the socket.
-- Fiber 2 calls `client.<<`, blocks, sets up an I/O watcher, and suspends,
-  switching to the next runnable fiber, fiber 3.
-- Fiber 3 resumes and calls `client.gets`. The call blocks, an I/O event watcher
-  is set up, and the fiber suspends.
-
-## Fiber states
-
-In Polyphony, each fiber has one of the following states at any given moment:
-
-- `:running`: this is the state of the currently running fiber.
-- `:dead`: the fiber has terminated.
-- `:waiting`: the fiber is suspended, waiting for something to wake it up.
-- `:runnable`: the fiber is on the run queue and will be run shortly.
-
 ## Fiber scheduling and fiber switching
 
 The Polyphony scheduling model makes a clear separation between the scheduling
 of fibers and the switching of fibers. The scheduling of fibers is the act of
-marking the fiber to be run at the earliest opportunity, but not immediately.
-The switching of fibers is the act of transferring control to another fiber, in
-this case the first fiber in the list of *currently* scheduled fibers.
+marking the fiber as `:runnable`, to be run at the earliest opportunity, but not
+immediately. The switching of fibers is the act of actually transferring control
+to another fiber, namely the first fiber in the run queue.
 
 The scheduling of fibers can occur at any time, either as a result of an event
 occuring, an exception being raised, or using `Fiber#schedule`. The switching of
-fibers will occur only when a blocking operation is started, or upon calling
-`Fiber#suspend` or `Fiber#snooze`. In order to switch to a scheduled fiber,
-Polyphony uses `Fiber#transfer`.
+fibers will occur only when the currently running fiber has reached a
+switchpoint, e.g. when a blocking operation is started, or upon calling
+`Fiber#suspend` or `Fiber#snooze`. As mentioned earlier, in order to switch to a
+scheduled fiber, Polyphony uses `Fiber#transfer`.
 
-When a fiber terminates, any other scheduled fibers will be run. If no fibers
+When a fiber terminates, any other runnable fibers will be run. If no fibers
 are waiting and the main fiber is done running, the Ruby process will terminate.
 
 ## Interrupting blocking operations
@@ -185,9 +173,42 @@ by scheduling the corresponding fiber with an exception:
 ```ruby
 def timeout(duration)
   fiber = Fiber.current
-  watcher = Gyro::Timer.new(duration) { fiber.transfer(TimerException.new) }
+  interrupter = spin do
+    Gyro::Timer.new(duration, 0).await
+    fiber.transfer(TimerException.new)
+  end
   yield
 ensure
-  watcher.active = false
+  interrupter.stop
 end
 ```
+
+## Fiber Scheduling in a Multithreaded Program
+
+Polyphony performs fiber scheduling separately for each thread. Each thread,
+therefore, will be able to run multiple fibers independently from other threads.
+
+## The fiber scheduling algorithm in full
+
+Here is the summary of the Polyphony scheduling algorithm:
+
+- loop
+  - pull first runnable fiber from run queue
+  - if runnable fiber is not nil
+    -  if the ref count greater than 0
+      - increment the run_no_wait counter
+      - if the run_no_wait counter is greater than 10 and greater than the run
+        queue length
+        - reset the run_no_wait counter
+        - run the event loop once without waiting for events (using
+          `EVRUN_NOWAIT`)
+    - break out of the loop
+  - if the ref count is 0
+    - break out of the loop
+  - run the event loop until one or more events are generated (using
+    `EVRUN_ONCE`)
+- if next runnable fiber is nil
+  - return
+- get scheduled resume value for next runnable fiber
+- mark the next runnable fiber as not runnable
+- switch to the next runnable fiber using `Fiber#transfer`

data/docs/user-guide/all-about-timers.md

@@ -0,0 +1,126 @@
+---
+layout: page
+title: All About Timers
+nav_order: 1
+parent: User Guide
+permalink: /user-guide/all-about-timers/
+---
+# All About Timers
+
+Timers form a major part of writing dynamic concurrent programs. They allow
+programmers to create delays and to perform recurring operations with a
+controllable frequency. Crucially, they also enable the implementation of
+timeouts, which are an important aspect of concurrent programming.
+
+## Sleeping
+
+Sometimes, your code needs to wait for a certain period of time. For example,
+implementing a retry mechanism for failed HTTP requests might involve waiting
+for a few seconds before retrying. Polyphony patches the `Kernel#sleep` method
+to be fiber-aware, that is to yield control of execution while waiting for a
+timer to elapse.
+
+```ruby
+# This is a naive retry implementation
+def fetch(url)
+  fetch_url(url)
+rescue
+  sleep 1
+  retry
+end
+```
+
+## Sleeping Forever
+
+The `#sleep` method can also be used to sleep forever, if no argument is given:
+
+```ruby
+puts "Go to sleep"
+sleep
+puts "Woke up" # this line will not be executed
+```
+
+The `#sleep` forever call can be used for example in the main fiber when we do
+all our work in other fibers, since once the main fiber terminates the program
+exits.
+
+## Doing Work Later 
+
+While `#sleep` allows you to block execution of the current fiber, sometimes you
+want to perform some work later, while not blocking the current fiber. This is done simply by spinning off another fiber:
+
+```ruby
+do_some_stuff
+spin do
+  sleep 3
+  do_stuff_later
+end
+do_some_more_stuff
+```
+
+## Using timeouts
+
+Polyphony provides the following global methods for using timeouts:
+
+- `#move_on_after` - used for cancelling an operation after a certain period of time without raising an exception:
+
+  ```ruby
+  move_on_after 1 do
+    sleep 60
+  end
+  ```
+
+  This method also takes an optional return value argument:
+
+  ```ruby
+  move_on_after 1, with_value: 'bar' do
+    sleep 60
+    'foo'
+  end #=> 'bar'
+  ```
+
+- `#cancel_after` - used for cancelling an operation after a certain period of time with a `Cancel` exception:
+
+  ```ruby
+  cancel_after 1 do
+    sleep 60
+  end #=> raises Cancel
+  ```
+
+Polyphony also provides a fiber-aware version of the core Ruby `Timeout` API, which may be used directly or indirectly to interrupt blocking operations.
+
+## Using Raw Timers
+
+Polyphony implements timers through the `Gyro::Timer` class, which encapsulates
+libev timer watchers. Using `Gyro::Timer` you can create both one-time and
+recurring timers:
+
+```ruby
+# Create a one-time timer
+one_time = Gyro::Timer.new(1, 0)
+
+# Create a recurring timer
+recurring = Gyro::Timer.new(0.5, 1.5)
+```
+
+Once your timer is created, you can wait for it using the `#await` method:
+
+```ruby
+def delay(duration)
+  timer = Gyro::Timer.new(duration, 0)
+  timer.await
+end
+```
+
+Waiting for the timer will *block* the current timer. This means that if you
+want to do other work while waiting for the timer, you need to put it on a
+different fiber:
+
+```ruby
+timer = Gyro::Timer.new(3, 0)
+spin {
+  sleep 3
+  do_something_else
+}
+do_blocking_operation
+```

data/docs/user-guide/web-server.md

@@ -37,7 +37,7 @@ Note that requests are handled using a callback block which takes a single
 argument. The `request` object provides the entire API for responding to the
 client.
 
-Each client connection will be handled in a separate coprocess, allowing
+Each client connection will be handled in a separate fiber, allowing
 concurrent processing of incoming requests.
 
 ## HTTP 2 support
@@ -47,7 +47,7 @@ HTTP 2 support is baked in to the server, which supports both HTTP 2 upgrades
 in a completely effortless manner.
 
 Since HTTP 2 connections are multiplexed, allowing multiple concurrent requests
-on a single connection, each HTTP 2 stream is handled in a separate coprocess.
+on a single connection, each HTTP 2 stream is handled in a separate fiber.
 
 ## TLS termination
 

data/docs/user-guide.md

@@ -6,5 +6,5 @@ has_children: true
 section: true
 has_toc: false
 nav_order: 4
-section_link: /user-guide/web-server
+section_link: /user-guide/all-about-timers
 ---

data/examples/core/xx-deferring-an-operation.rb

@@ -3,9 +3,9 @@
 require 'bundler/setup'
 require 'polyphony'
 
-defer do
+spin do
   puts 'two'
-  defer { puts 'four' }
+  spin { puts 'four' }
   puts 'three'
 end
 

data/examples/core/xx-sleep-forever.rb

@@ -0,0 +1,9 @@
+# frozen_string_literal: true
+
+require 'bundler/setup'
+require 'polyphony'
+
+Exception.__disable_sanitized_backtrace__ = true
+
+puts "press Ctrl-C!"
+sleep

data/examples/core/xx-snooze-starve.rb

@@ -0,0 +1,16 @@
+# frozen_string_literal: true
+
+require 'bundler/setup'
+require 'polyphony'
+
+Exception.__disable_sanitized_backtrace__ = true
+
+f = spin_loop {
+  snooze
+}
+
+puts 'going to sleep...'
+sleep 1
+puts 'woke up'
+f.stop
+

data/examples/core/xx-spin_error_backtrace.rb

@@ -9,7 +9,7 @@ end
 
 def deferred_error(t)
   puts "deferred_error"
-  defer { de2(t) }
+  spin { de2(t) }
 end
 
 def de2(t)

data/examples/core/xx-trace.rb

@@ -5,13 +5,12 @@ require 'polyphony'
 
 Exception.__disable_sanitized_backtrace__ = true
 
-Gyro.trace(true)
 
 sleep 0
 $records = []
 
+Gyro.trace(true)
 trace = Polyphony::Trace.new { |r| $records << r }
-
 trace.enable
 
 f2 = spin(:f2) { 3.times { sleep 0.1 } }

data/examples/core/xx-worker-thread.rb

@@ -0,0 +1,30 @@
+# frozen_string_literal: true
+
+require 'bundler/setup'
+require 'polyphony'
+
+def do_work(client)
+  result = yield
+  client.schedule(result)
+rescue Exception => e
+  client.schedule(e)
+end
+
+$worker = Thread.new do
+  Fiber.current.tag = :worker
+  loop do
+    client, block = receive
+    do_work(client, &block)
+  end
+rescue Exception => e
+  p e
+end
+
+def process(&block)
+  $worker.main_fiber << [Fiber.current, block]
+  receive
+end
+
+sleep 0.1
+
+p process { 1 + 1 }

data/examples/io/xx-happy-eyeballs.rb

@@ -0,0 +1,37 @@
+# frozen_string_literal: true
+
+# idea taken from the example given in trio:
+# https://www.youtube.com/watch?v=oLkfnc_UMcE
+
+require 'bundler/setup'
+require 'polyphony'
+
+def try_connect(target, supervisor)
+  puts "trying #{target[2]}"
+  socket = Polyphony::Net.tcp_connect(target[2], 80)
+  # connection successful
+  supervisor.stop([target[2], socket])
+rescue IOError, SystemCallError
+  # ignore error
+end
+
+def happy_eyeballs(hostname, port, max_wait_time: 0.025)
+  targets = Socket.getaddrinfo(hostname, port, :INET, :STREAM)
+  t0 = Time.now
+  cancel_after(5) do
+    success = supervise do |supervisor|
+      targets.each_with_index do |t, idx|
+        sleep(max_wait_time) if idx > 0
+        supervisor.spin { try_connect(t, supervisor) }
+      end
+    end
+    if success
+      puts format('success: %s (%.3fs)', success[0], Time.now - t0)
+    else
+      puts "timed out (#{Time.now - t0}s)"
+    end
+  end
+end
+
+# Let's try it out:
+happy_eyeballs('debian.org', 'https')

data/ext/gyro/gyro.c

@@ -11,6 +11,7 @@ ID ID_inspect;
 ID ID_new;
 ID ID_raise;
 ID ID_ivar_running;
+ID ID_ivar_thread;
 ID ID_runnable;
 ID ID_runnable_value;
 ID ID_size;
@@ -117,10 +118,13 @@ static VALUE Fiber_state(VALUE self) {
 }
 
 inline void Gyro_schedule_fiber(VALUE fiber, VALUE value) {
-  if (__tracing_enabled__) {
-    rb_funcall(rb_cObject, ID_fiber_trace, 3, SYM_fiber_schedule, fiber, value);
+  VALUE thread = rb_ivar_get(fiber, ID_ivar_thread);
+  if (thread != Qnil) {
+    Thread_schedule_fiber(thread, fiber, value);
+  }
+  else {
+    rb_warn("No thread set for fiber");
   }
-  Thread_schedule_fiber(rb_thread_current(), fiber, value);
 }
 
 VALUE Gyro_trace(VALUE self, VALUE enabled) {
@@ -154,6 +158,7 @@ void Init_Gyro() {
   ID_empty          = rb_intern("empty?");
   ID_inspect        = rb_intern("inspect");
   ID_ivar_running   = rb_intern("@running");
+  ID_ivar_thread    = rb_intern("@thread");
   ID_new            = rb_intern("new");
   ID_pop            = rb_intern("pop");
   ID_push           = rb_intern("push");

data/ext/gyro/gyro.h

@@ -27,7 +27,8 @@ VALUE IO_read_watcher(VALUE io);
 VALUE IO_write_watcher(VALUE io);
 VALUE Gyro_IO_await(VALUE self);
 
-VALUE Gyro_Selector_run(VALUE self);
+VALUE Gyro_Selector_run(VALUE self, VALUE current_fiber);
+void Gyro_Selector_run_no_wait(VALUE self, VALUE current_fiber, long runnable_count);
 
 VALUE Gyro_Timer_await(VALUE self);
 
@@ -36,6 +37,7 @@ void io_set_read_length(VALUE str, long n, int shrinkable);
 VALUE io_enc_str(VALUE str, rb_io_t *fptr);
 
 struct ev_loop *Gyro_Selector_ev_loop(VALUE selector);
+ev_tstamp Gyro_Selector_now(VALUE selector);
 struct ev_loop *Gyro_Selector_current_thread_ev_loop();
 long Gyro_Selector_pending_count(VALUE self);
 
@@ -50,6 +52,9 @@ VALUE Thread_post_fork(VALUE thread);
 
 #define OBJ_ID(obj) (NUM2LONG(rb_funcall(obj, rb_intern("object_id"), 0)))
 #define INSPECT(...) (rb_funcall(rb_cObject, rb_intern("p"), __VA_ARGS__))
+#define FIBER_TRACE(...) if (__tracing_enabled__) { \
+  rb_funcall(rb_cObject, ID_fiber_trace, __VA_ARGS__); \
+}
 
 extern VALUE mGyro;
 extern VALUE cGyro_Async;
@@ -65,6 +70,7 @@ extern ID ID_each;
 extern ID ID_fiber_trace;
 extern ID ID_inspect;
 extern ID ID_ivar_running;
+extern ID ID_ivar_thread;
 extern ID ID_new;
 extern ID ID_raise;
 extern ID ID_runnable;

data/ext/gyro/queue.c

@@ -78,22 +78,50 @@ VALUE Gyro_Queue_shift(VALUE self) {
   return Gyro_Async_await(async);
 }
 
+VALUE Gyro_Queue_shift_no_wait(VALUE self) {
+  struct Gyro_Queue *queue;
+  GetGyro_Queue(self, queue);
+
+  return rb_ary_shift(queue->queue);
+}
+
 VALUE Gyro_Queue_shift_all(VALUE self) {
   struct Gyro_Queue *queue;
   GetGyro_Queue(self, queue);
 
+  VALUE old_queue = queue->queue;
+  queue->queue = rb_ary_new();
+
   if (rb_block_given_p()) {
-    while (RARRAY_LEN(queue->queue) > 0) {
-      rb_yield(rb_ary_shift(queue->queue));
+    long len = RARRAY_LEN(old_queue);
+    for (long i = 0; i < len; i++) {
+      rb_yield(RARRAY_AREF(old_queue, i));
     }
+    // while (RARRAY_LEN(old_queue) > 0) {
+    //   rb_yield(rb_ary_shift(old_queue));
+    // }
+    return self;
   }
   else {
-    rb_ary_clear(queue->queue);
+    return old_queue;
   }
+}
+
+VALUE Gyro_Queue_clear(VALUE self) {
+  struct Gyro_Queue *queue;
+  GetGyro_Queue(self, queue);
 
+  rb_ary_clear(queue->queue);
   return self;
 }
 
+VALUE Gyro_Queue_empty_p(VALUE self) {
+  struct Gyro_Queue *queue;
+  GetGyro_Queue(self, queue);
+
+  return (RARRAY_LEN(queue->queue) == 0) ? Qtrue : Qfalse;
+}
+
 void Init_Gyro_Queue() {
   cGyro_Queue = rb_define_class_under(mGyro, "Queue", rb_cData);
   rb_define_alloc_func(cGyro_Queue, Gyro_Queue_allocate);
@@ -104,6 +132,10 @@ void Init_Gyro_Queue() {
 
   rb_define_method(cGyro_Queue, "pop", Gyro_Queue_shift, 0);
   rb_define_method(cGyro_Queue, "shift", Gyro_Queue_shift, 0);
+  
+  rb_define_method(cGyro_Queue, "shift_no_wait", Gyro_Queue_shift_no_wait, 0);
 
   rb_define_method(cGyro_Queue, "shift_each", Gyro_Queue_shift_all, 0);
+  rb_define_method(cGyro_Queue, "clear", Gyro_Queue_clear, 0);
+  rb_define_method(cGyro_Queue, "empty?", Gyro_Queue_empty_p, 0);
 }