polyphony 0.33 → 0.34

Sign up to get free protection for your applications and to get access to all the features.
Files changed (49) hide show
  1. checksums.yaml +4 -4
  2. data/CHANGELOG.md +8 -0
  3. data/Gemfile.lock +1 -1
  4. data/TODO.md +93 -68
  5. data/bin/polyphony-debug +87 -0
  6. data/docs/_includes/nav.html +5 -1
  7. data/docs/_sass/overrides.scss +4 -1
  8. data/docs/api-reference.md +11 -0
  9. data/docs/api-reference/exception.md +27 -0
  10. data/docs/api-reference/fiber.md +407 -0
  11. data/docs/api-reference/io.md +36 -0
  12. data/docs/api-reference/object.md +99 -0
  13. data/docs/api-reference/polyphony-baseexception.md +33 -0
  14. data/docs/api-reference/polyphony-cancel.md +26 -0
  15. data/docs/api-reference/polyphony-moveon.md +24 -0
  16. data/docs/api-reference/polyphony-net.md +20 -0
  17. data/docs/api-reference/polyphony-process.md +28 -0
  18. data/docs/api-reference/polyphony-resourcepool.md +59 -0
  19. data/docs/api-reference/polyphony-restart.md +18 -0
  20. data/docs/api-reference/polyphony-terminate.md +18 -0
  21. data/docs/api-reference/polyphony-threadpool.md +67 -0
  22. data/docs/api-reference/polyphony-throttler.md +77 -0
  23. data/docs/api-reference/polyphony.md +36 -0
  24. data/docs/api-reference/thread.md +88 -0
  25. data/docs/getting-started/tutorial.md +59 -156
  26. data/docs/index.md +2 -0
  27. data/examples/core/forever_sleep.rb +19 -0
  28. data/examples/core/xx-caller.rb +12 -0
  29. data/examples/core/xx-exception-backtrace.rb +40 -0
  30. data/examples/core/xx-fork-spin.rb +42 -0
  31. data/examples/core/xx-spin-fork.rb +49 -0
  32. data/examples/core/xx-supervise-process.rb +30 -0
  33. data/ext/gyro/gyro.h +1 -0
  34. data/ext/gyro/selector.c +8 -0
  35. data/ext/gyro/thread.c +8 -2
  36. data/lib/polyphony.rb +64 -17
  37. data/lib/polyphony/adapters/process.rb +29 -0
  38. data/lib/polyphony/adapters/trace.rb +6 -4
  39. data/lib/polyphony/core/exceptions.rb +5 -0
  40. data/lib/polyphony/core/global_api.rb +15 -0
  41. data/lib/polyphony/extensions/fiber.rb +89 -59
  42. data/lib/polyphony/version.rb +1 -1
  43. data/test/test_fiber.rb +23 -75
  44. data/test/test_global_api.rb +39 -0
  45. data/test/test_kernel.rb +5 -7
  46. data/test/test_process_supervision.rb +46 -0
  47. data/test/test_signal.rb +2 -3
  48. data/test/test_supervise.rb +103 -0
  49. metadata +29 -2
@@ -0,0 +1,36 @@
1
+ ---
2
+ layout: page
3
+ title: ::IO
4
+ parent: API Reference
5
+ permalink: /api-reference/io/
6
+ ---
7
+ # ::IO
8
+
9
+ [Ruby core IO documentation](https://ruby-doc.org/core-2.7.0/IO.html)
10
+
11
+ Polyphony reimplements a significant number of IO class and instance methods to
12
+ be fiber-aware. Polyphony also adds methods for accessing the associated event
13
+ watchers.
14
+
15
+ ## Class Methods
16
+
17
+ ## Instance methods
18
+
19
+ ### #read_watcher → io_watcher
20
+
21
+ Returns the read watcher associated with the IO. The watcher is automatically
22
+ created and cached. The watcher is an instance of `Gyro::IO`. Normally this
23
+ method is not called directly from application code.
24
+
25
+ ```ruby
26
+ def read_ten_chars(io)
27
+ io.read_watcher.await
28
+ io.read(10)
29
+ end
30
+ ```
31
+
32
+ ### #write_watcher → io_watcher
33
+
34
+ Returns the write watcher associated with the IO. The watcher is automatically
35
+ created and cached. The watcher is an instance of `Gyro::IO`. Normally this
36
+ method is not called directly from application code.
@@ -0,0 +1,99 @@
1
+ ---
2
+ layout: page
3
+ title: ::Object (Global API)
4
+ parent: API Reference
5
+ permalink: /api-reference/object/
6
+ ---
7
+ # ::Object (Global API)
8
+
9
+ The global Polyphony API is designed to feel almost like a part of the Ruby
10
+ runtime. The global API contains multiple methods for creating and controlling
11
+ fibers, as well as miscellaneous methods for dealing with timers and other
12
+ events, with minimal boilerplate. The API is implemented as a module included in
13
+ the `Object` class, allowing access from any receiver.
14
+
15
+ ### #after(interval, { block }) → fiber
16
+
17
+ Run the given block after the given time interval (specified in seconds). This
18
+ method spins up a separate fiber that will sleep for the given interval, then
19
+ run the given block.
20
+
21
+ ```ruby
22
+ f = spin { do_some_big_work }
23
+ after(1) { f.stop }
24
+ f.await
25
+ ```
26
+
27
+ ### #cancel_after(interval, { block }) → object
28
+
29
+ Run the given block, cancelling it after the given time interval by raising a
30
+ `Polyphony::Cancel` exception. If uncaught, the exception will be propagated.
31
+
32
+ ```ruby
33
+ spin do
34
+ cancel_after(3) { do_some_work }
35
+ rescue Polyphony::Cancel
36
+ puts "work was cancelled"
37
+ end
38
+ ```
39
+
40
+ ### #every(interval, { block }) → object
41
+
42
+ Runs the given block repeatedly, at the given time interval. This method will
43
+ block until an exception is raised.
44
+
45
+ ```ruby
46
+ every(3) do
47
+ puts "I'm still alive"
48
+ end
49
+ ```
50
+
51
+ ### #move_on_after(interval, with_value: nil, { block }) → object
52
+
53
+ Run the given block, interrupting it after the given time interval by raising a
54
+ `Polyphony::MoveOn` exception. The `with_value` keyword argument can be used to
55
+ set the value returned from the block if the timeout has elapsed.
56
+
57
+ ```ruby
58
+ result = move_on_after(3, with_value: 'bar') { sleep 5; 'foo' }
59
+ result #=> 'bar'
60
+ ```
61
+
62
+ ### #receive → object
63
+
64
+ Shortcut for `Fiber.current.receive`
65
+
66
+ ### #receive_pending → [*object]
67
+
68
+ Shortcut for `Fiber.current.receive_pending`
69
+
70
+ ### #sleep(duration = nil) → fiber
71
+
72
+ Sleeps for the given duration.
73
+
74
+ ### #spin(tag = nil, { block}) → fiber
75
+
76
+ Shortcut for `Fiber.current.spin`
77
+
78
+ ### #spin_loop(tag = nil, rate: nil, &block) → fiber
79
+
80
+ Spins up a new fiber that runs the given block in a loop. If `rate` is given,
81
+ the loop is throttled to run `rate` times per second.
82
+
83
+ ```ruby
84
+ # print twice a second
85
+ f = spin_loop(rate: 2) { puts 'hello world' }
86
+ sleep 2
87
+ f.stop
88
+ ```
89
+
90
+ ### #throttled_loop(rate, count: nil, &block) → object
91
+
92
+ Runs the given block in a loop at the given rate (times per second). If `count`
93
+ is given, the loop will be run for the specified number of times and then
94
+ returns. Otherwise, the loop is infinite (unless an exception is raised).
95
+
96
+ ```ruby
97
+ # twice a second
98
+ throttled_loop(2) { puts 'hello world' }
99
+ ```
@@ -0,0 +1,33 @@
1
+ ---
2
+ layout: page
3
+ title: Polyphony::BaseException
4
+ parent: API Reference
5
+ permalink: /api-reference/polyphony-baseexception/
6
+ ---
7
+ # Polyphony::BaseException
8
+
9
+ The `Polyphony::BaseException` is a common base class for exceptions used to
10
+ control fiber execution. Instances of descendant classes are meant to be created
11
+ explicitly using `new`, e.g. `Polyphony::MoveOn.new`, rather than using `raise
12
+ Polyphony::MoveOn`. Normally an application will not use those classes directly
13
+ but would rather use APIs such as `Fiber#interrupt`.
14
+
15
+ ## Derived classes
16
+
17
+ - [`Polyphony::Cancel`](../polyphony-cancel/)
18
+ - [`Polyphony::MoveOn`](../polyphony-moveon/)
19
+ - [`Polyphony::Restart`](../polyphony-restart/)
20
+ - [`Polyphony::Terminate`](../polyphony-terminate/)
21
+
22
+ ## Instance methods
23
+
24
+ ### #initialize(value = nil)
25
+
26
+ Initializes the exception with an optional result value. The value will be used
27
+ as the result of the block being interrupted or the fiber being terminated.
28
+
29
+ ```ruby
30
+ f = spin { 'foo' }
31
+ f.raise(Polyphony::Terminate.new('bar'))
32
+ f.await #=> 'bar'
33
+ ```
@@ -0,0 +1,26 @@
1
+ ---
2
+ layout: page
3
+ title: Polyphony::Cancel
4
+ parent: API Reference
5
+ permalink: /api-reference/polyphony-cancel/
6
+ ---
7
+ # Polyphony::Cancel
8
+
9
+ `Polyphony::Cancel` is an exception class used to interrupt a blocking operation
10
+ with an exception that must be rescued. This exception is will propagate if not
11
+ rescued. A `Polyphony::Cancel` exception is normally raised using APIs such as
12
+ `Fiber#cancel!` or `Object#cancel_after`.
13
+
14
+ ```ruby
15
+ require 'httparty'
16
+ require 'time'
17
+
18
+ def current_server_time
19
+ cancel_after(10) do
20
+ response_body = HTTParty.get(TIME_URL).body
21
+ Time.parse(response_body)
22
+ end
23
+ rescue Polyphony::Cancel
24
+ Time.now
25
+ end
26
+ ```
@@ -0,0 +1,24 @@
1
+ ---
2
+ layout: page
3
+ title: Polyphony::MoveOn
4
+ parent: API Reference
5
+ permalink: /api-reference/polyphony-moveon/
6
+ ---
7
+ # Polyphony::MoveOn
8
+
9
+ `Polyphony::MoveOn` is an exception class used to interrupt a blocking operation
10
+ without propagating the excception. A `Polyphony::MoveOn` exception is normally
11
+ raised using APIs such as `Fiber#interrupt` or `Object#move_on_after`. This
12
+ exception allows you to set the result of the operation being interrupted.
13
+
14
+ ```ruby
15
+
16
+ def do_something_slow
17
+ sleep 10
18
+ 'foo'
19
+ end
20
+
21
+ f = spin { do_something_slow }
22
+ f.interrupt('bar')
23
+ f.await #=> 'bar'
24
+ ```
@@ -0,0 +1,20 @@
1
+ ---
2
+ layout: page
3
+ title: Polyphony::Net
4
+ parent: API Reference
5
+ permalink: /api-reference/polyphony-net/
6
+ ---
7
+ # Polyphony::Net
8
+
9
+ The `Polyphony::Net` provides convenience methods for working with sockets. The
10
+ module unifies secure and non-secure socket APIs.
11
+
12
+ ## Class Methods
13
+
14
+ ### #tcp_connect(host, port, opts = {}) → socket
15
+
16
+ Connects to a TCP server.
17
+
18
+ ### #tcp_listen(host = nil, port = nil, opts = {}) → socket
19
+
20
+ Opens a server socket for listening to incoming connections.
@@ -0,0 +1,28 @@
1
+ ---
2
+ layout: page
3
+ title: Polyphony::Process
4
+ parent: API Reference
5
+ permalink: /api-reference/polyphony-process/
6
+ ---
7
+ # Polyphony::Process
8
+
9
+ The `Polyphony::Process` module is used to watch child processes.
10
+
11
+ ## Class Methods
12
+
13
+ ### #watch(cmd = nil, { block })
14
+
15
+ Starts a child process, blocking until the child process terminates. If `#watch`
16
+ is interrupted before the child process terminates, the child process is sent a
17
+ `TERM` signal, and awaited. After 5 seconds, if the child has still not
18
+ terminated, it will be sent a `KILL` signal and awaited. This method is normally
19
+ used in conjunction with `#supervise` in order to supervise child processes.
20
+
21
+ If `cmd` is given, the child process is started using `Kernel#spawn` running a
22
+ shell command. If a block is given, the child process is started using
23
+ [`Polyphony#fork`](../polyphony/#fork-block---pid).
24
+
25
+ ```ruby
26
+ Polyphony::Process.watch('echo "Hello World"; sleep 1')
27
+ supervise(restart: :always)
28
+ ```
@@ -0,0 +1,59 @@
1
+ ---
2
+ layout: page
3
+ title: Polyphony::ResourcePool
4
+ parent: API Reference
5
+ permalink: /api-reference/polyphony-resourcepool/
6
+ ---
7
+ # Polyphony::ResourcePool
8
+
9
+ `Polyphony::ResourcePool` implements a general purpose resource pool for
10
+ limiting concurrent access to a resource or multiple copies thereof. A resource
11
+ pool might be used for example to limit the number of concurrent database
12
+ connections.
13
+
14
+ ## Class methods
15
+
16
+ ## Instance methods
17
+
18
+ ### #acquire({ block })
19
+
20
+ Acquires a resource and passes it to the given block. The resource will be used
21
+ exclusively by the given block, and then returned to the pool. This method
22
+ blocks until the given block has completed running. If no resource is available,
23
+ this method blocks until a resource has been released.
24
+
25
+ ```ruby
26
+ db_connections = Polyphony::ResourcePool.new(limit: 5) { PG.connect(opts) }
27
+
28
+ def query_records(sql)
29
+ db_connections.acquire do |db|
30
+ db.query(sql).to_a
31
+ end
32
+ end
33
+ ```
34
+
35
+ ### #available → count
36
+
37
+ Returns the number of resources currently available in the resource pool.
38
+
39
+ ### #initialize(limit: number, { block })
40
+
41
+ Initializes a new resource pool with the given maximum number of concurrent
42
+ resources. The given block is used to create the resource.
43
+
44
+ ```ruby
45
+ require 'postgres'
46
+
47
+ opts = { host: '/tmp', user: 'admin', dbname: 'mydb' }
48
+ db_connections = Polyphony::ResourcePool.new(limit: 5) { PG.connect(opts) }
49
+ ```
50
+
51
+ ### #limit → count
52
+
53
+ Returns the size limit of the resource pool.
54
+
55
+ ### #size → count
56
+
57
+ Returns the total number of allocated resources in the resource pool. This
58
+ includes both available and unavailable resources.
59
+
@@ -0,0 +1,18 @@
1
+ ---
2
+ layout: page
3
+ title: Polyphony::Restart
4
+ parent: API Reference
5
+ permalink: /api-reference/polyphony-restart/
6
+ ---
7
+ # Polyphony::Restart
8
+
9
+ `Polyphony::Restart` is an exception class used to restart a fiber. Applications
10
+ will not normally raise a `Polyphony::Restart` exception, but would rather use
11
+ `Fiber#restart`.
12
+
13
+ ```ruby
14
+ f = spin { do_something_slow }
15
+ ...
16
+ f.restart
17
+ ...
18
+ ```
@@ -0,0 +1,18 @@
1
+ ---
2
+ layout: page
3
+ title: Polyphony::Terminate
4
+ parent: API Reference
5
+ permalink: /api-reference/polyphony-terminate/
6
+ ---
7
+ # Polyphony::Terminate
8
+
9
+ `Polyphony::Terminate` is an exception class used to terminate a fiber without
10
+ propagating the exception. It should never be rescued. A `Polyphony::Terminate`
11
+ exception is normally raised using APIs such as `Fiber#terminate` or
12
+ `Fiber#terminate_all_children`.
13
+
14
+ ```ruby
15
+ f = spin { do_something_slow }
16
+ ...
17
+ f.terminate
18
+ ```
@@ -0,0 +1,67 @@
1
+ ---
2
+ layout: page
3
+ title: Polyphony::ThreadPool
4
+ parent: API Reference
5
+ permalink: /api-reference/polyphony-threadpool/
6
+ ---
7
+ # Polyphony::ThreadPool
8
+
9
+ `Polyphony::ThreadPool` implements a general purpose thread pool, normally used
10
+ for the execution of non-fiber aware operations, such as C-extension based
11
+ third-party libraries or other system call blocking APIs. The Polyphony
12
+ implementation of a thread pool allows limiting the number of threads used for
13
+ performing a recurring operation across one or more fibers.
14
+
15
+ A default thread pool is available for quick access to this feature.
16
+
17
+ ## Class methods
18
+
19
+ ### #process({ block }) → object
20
+
21
+ Runs the given block on the default thread pool. The default pool will be
22
+ created on the first call to `#process`. This method will block until the
23
+ operation has completed. The return value is that of the given block. Any
24
+ uncaught exception will be propagated to the callsite.
25
+
26
+ ```ruby
27
+ result = Polyphony::ThreadPool.process { lengthy_op }
28
+ ```
29
+
30
+ ## Instance methods
31
+
32
+ ### #busy? → true or false
33
+
34
+ Returns true if operations are currently being run on the thread pool.
35
+
36
+ ### cast({ block }) → pool
37
+
38
+ Runs the given block on one of the threads in the pool in a fire-and-forget
39
+ manner, without waiting for the operation to complete. Using `#cast` to run an
40
+ operation means there's no way of knowing if the operation has completed or if
41
+ any exception has been raised, other than inside the block.
42
+
43
+ ```ruby
44
+ my_pool.cast { puts 'Hello world' }
45
+ do_something_else
46
+ ```
47
+
48
+ ### #initialize(size = Etc.nprocessors)
49
+
50
+ Initializes a new instance of `Polyphony::ThreadPool` with the given maximum
51
+ number of threads. The default size is the number of available processors
52
+ (number of CPU cores).
53
+
54
+ ```ruby
55
+ my_pool = Polyphony::ThreadPool.new(3)
56
+ ```
57
+
58
+ ### #process({ block }) → object
59
+
60
+ Runs the given block on one of the threads in the thread pool and blocks until
61
+ the operation has completed. The return value is that of the given block. Any
62
+ uncaught exception will be propagated to the callsite.
63
+
64
+ ```ruby
65
+ pool = Polyphony::ThreadPool.new(3)
66
+ result = pool.process { lengthy_op }
67
+ ```
@@ -0,0 +1,77 @@
1
+ ---
2
+ layout: page
3
+ title: Polyphony::Throttler
4
+ parent: API Reference
5
+ permalink: /api-reference/polyphony-throttler/
6
+ ---
7
+ # Polyphony::Throttler
8
+
9
+ `Polyphony::Throttler` implements general purpose operation throttling, or rate
10
+ limiting. A `Polyphony::Throttler` instance may be used to limit the rate of an
11
+ arbitrary operation in a single fiber, or across multiple fibers. For example,
12
+ an HTTP server can limit the number of requests per second across for each
13
+ client, or for all clients.
14
+
15
+ A throttler is invoked using its `#call` method, e.g.:
16
+
17
+ ```ruby
18
+ # throttle rate: one per second
19
+ throttler = Polyphony::Throttler.new(1)
20
+
21
+ 10.times do |i|
22
+ spin_loop { throttler.call { p [i, Time.now] } }
23
+ end
24
+ ```
25
+
26
+ If many throttler instances are created over the application's lifetime, they
27
+ should be stopped using the `#stop` method in order to prevent memory leaks.
28
+ This is best done using an `ensure` block:
29
+
30
+ ```ruby
31
+ def start_server
32
+ throttler = Polyphony::Throttler.new(1000)
33
+ MyServer.start do |req|
34
+ throttler.call { handle_request(req) }
35
+ end
36
+ ensure
37
+ throttler.stop
38
+ end
39
+ ```
40
+
41
+ ## Instance methods
42
+
43
+ ### #initialize(rate)<br>#initialize(interval: interval)<br>#initialize(rate: rate)
44
+
45
+ Initializes the throttler with the given rate. The rate can be specified either
46
+ as a number signifying the maximum rate per second, or as a keyword argument. If
47
+ the rate is specified using the `interval:` keyword argument, the value given is
48
+ the minimum interval between consecutive invocations.
49
+
50
+ ```ruby
51
+ # These are all equivalent
52
+ Polyphony::Throttler.new(10)
53
+ Polyphony::Throttler.new(rate: 10)
54
+ Polyphony::Throttler.new(interval: 0.1)
55
+ ```
56
+
57
+ ### #call({ block }) → object
58
+
59
+ Invokes the throttler with the given block. This method will sleep for an
60
+ interval of time required to throttle the execution of the given block. The
61
+ return value is the return value of the given block.
62
+
63
+ ### #stop → throttler
64
+
65
+ Stops the timer associated with the throttler. This method should be called when
66
+ the throttler is no longer needed. This is best done from an `ensure` block.
67
+
68
+ ```ruby
69
+ def start_server
70
+ throttler = Polyphony::Throttler.new(1000)
71
+ MyServer.start do |req|
72
+ throttler.call { handle_request(req) }
73
+ end
74
+ ensure
75
+ throttler.stop
76
+ end
77
+ ```