polyphony 0.99 → 0.99.2
Sign up to get free protection for your applications and to get access to all the features.
- checksums.yaml +4 -4
- data/.github/FUNDING.yml +1 -1
- data/.rubocop.yml +3 -3
- data/.yardopts +30 -0
- data/CHANGELOG.md +4 -0
- data/LICENSE +1 -1
- data/README.md +63 -29
- data/Rakefile +1 -5
- data/TODO.md +0 -4
- data/docs/{main-concepts/concurrency.md → concurrency.md} +2 -9
- data/docs/{main-concepts/design-principles.md → design-principles.md} +3 -9
- data/docs/{main-concepts/exception-handling.md → exception-handling.md} +2 -9
- data/docs/{main-concepts/extending.md → extending.md} +2 -9
- data/docs/faq.md +3 -16
- data/docs/{main-concepts/fiber-scheduling.md → fiber-scheduling.md} +1 -9
- data/docs/link_rewriter.rb +16 -0
- data/docs/{getting-started/overview.md → overview.md} +1 -30
- data/docs/{getting-started/tutorial.md → tutorial.md} +3 -28
- data/docs/{_posts/2020-07-26-polyphony-0.44.md → whats-new.md} +3 -1
- data/examples/adapters/redis_client.rb +3 -2
- data/examples/io/echo_server.rb +1 -1
- data/examples/io/echo_server_plain_ruby.rb +26 -0
- data/ext/polyphony/backend_io_uring.c +154 -9
- data/ext/polyphony/backend_io_uring_context.c +21 -12
- data/ext/polyphony/backend_io_uring_context.h +12 -7
- data/ext/polyphony/backend_libev.c +1 -1
- data/ext/polyphony/extconf.rb +24 -8
- data/ext/polyphony/fiber.c +79 -2
- data/ext/polyphony/io_extensions.c +53 -0
- data/ext/polyphony/pipe.c +42 -2
- data/ext/polyphony/polyphony.c +345 -31
- data/ext/polyphony/polyphony.h +9 -2
- data/ext/polyphony/queue.c +181 -0
- data/ext/polyphony/ring_buffer.c +0 -1
- data/ext/polyphony/runqueue.c +8 -1
- data/ext/polyphony/runqueue_ring_buffer.c +13 -0
- data/ext/polyphony/runqueue_ring_buffer.h +2 -1
- data/ext/polyphony/socket_extensions.c +6 -0
- data/ext/polyphony/thread.c +34 -2
- data/lib/polyphony/adapters/process.rb +11 -1
- data/lib/polyphony/adapters/sequel.rb +1 -1
- data/lib/polyphony/core/channel.rb +2 -0
- data/lib/polyphony/core/debug.rb +1 -1
- data/lib/polyphony/core/global_api.rb +25 -24
- data/lib/polyphony/core/resource_pool.rb +7 -6
- data/lib/polyphony/core/sync.rb +2 -2
- data/lib/polyphony/core/thread_pool.rb +3 -3
- data/lib/polyphony/core/timer.rb +8 -8
- data/lib/polyphony/extensions/exception.rb +2 -0
- data/lib/polyphony/extensions/fiber.rb +15 -13
- data/lib/polyphony/extensions/io.rb +127 -5
- data/lib/polyphony/extensions/kernel.rb +20 -2
- data/lib/polyphony/extensions/openssl.rb +100 -11
- data/lib/polyphony/extensions/pipe.rb +103 -7
- data/lib/polyphony/extensions/process.rb +13 -1
- data/lib/polyphony/extensions/socket.rb +93 -27
- data/lib/polyphony/extensions/thread.rb +9 -1
- data/lib/polyphony/extensions/timeout.rb +1 -1
- data/lib/polyphony/version.rb +2 -1
- data/lib/polyphony.rb +27 -7
- data/polyphony.gemspec +1 -8
- data/test/stress.rb +1 -1
- data/test/test_global_api.rb +45 -7
- data/test/test_socket.rb +96 -0
- data/test/test_timer.rb +5 -5
- metadata +17 -40
- data/docs/_config.yml +0 -64
- data/docs/_includes/head.html +0 -40
- data/docs/_includes/title.html +0 -1
- data/docs/_sass/custom/custom.scss +0 -10
- data/docs/_sass/overrides.scss +0 -0
- data/docs/api-reference/exception.md +0 -31
- data/docs/api-reference/fiber.md +0 -425
- data/docs/api-reference/index.md +0 -9
- data/docs/api-reference/io.md +0 -36
- data/docs/api-reference/object.md +0 -99
- data/docs/api-reference/polyphony-baseexception.md +0 -33
- data/docs/api-reference/polyphony-cancel.md +0 -26
- data/docs/api-reference/polyphony-moveon.md +0 -24
- data/docs/api-reference/polyphony-net.md +0 -20
- data/docs/api-reference/polyphony-process.md +0 -28
- data/docs/api-reference/polyphony-resourcepool.md +0 -59
- data/docs/api-reference/polyphony-restart.md +0 -18
- data/docs/api-reference/polyphony-terminate.md +0 -18
- data/docs/api-reference/polyphony-threadpool.md +0 -67
- data/docs/api-reference/polyphony-throttler.md +0 -77
- data/docs/api-reference/polyphony.md +0 -36
- data/docs/api-reference/thread.md +0 -88
- data/docs/favicon.ico +0 -0
- data/docs/getting-started/index.md +0 -10
- data/docs/getting-started/installing.md +0 -34
- /data/{docs/assets/img → assets}/echo-fibers.svg +0 -0
- /data/{docs → assets}/polyphony-logo.png +0 -0
- /data/{docs/assets/img → assets}/sleeping-fiber.svg +0 -0
@@ -1,59 +0,0 @@
|
|
1
|
-
---
|
2
|
-
layout: page
|
3
|
-
title: Polyphony::ResourcePool
|
4
|
-
parent: API Reference
|
5
|
-
permalink: /api-reference/polyphony-resourcepool/
|
6
|
-
---
|
7
|
-
# Polyphony::ResourcePool
|
8
|
-
|
9
|
-
`Polyphony::ResourcePool` implements a general purpose resource pool for
|
10
|
-
limiting concurrent access to a resource or multiple copies thereof. A resource
|
11
|
-
pool might be used for example to limit the number of concurrent database
|
12
|
-
connections.
|
13
|
-
|
14
|
-
## Class methods
|
15
|
-
|
16
|
-
## Instance methods
|
17
|
-
|
18
|
-
### #acquire({ block })
|
19
|
-
|
20
|
-
Acquires a resource and passes it to the given block. The resource will be used
|
21
|
-
exclusively by the given block, and then returned to the pool. This method
|
22
|
-
blocks until the given block has completed running. If no resource is available,
|
23
|
-
this method blocks until a resource has been released.
|
24
|
-
|
25
|
-
```ruby
|
26
|
-
db_connections = Polyphony::ResourcePool.new(limit: 5) { PG.connect(opts) }
|
27
|
-
|
28
|
-
def query_records(sql)
|
29
|
-
db_connections.acquire do |db|
|
30
|
-
db.query(sql).to_a
|
31
|
-
end
|
32
|
-
end
|
33
|
-
```
|
34
|
-
|
35
|
-
### #available → count
|
36
|
-
|
37
|
-
Returns the number of resources currently available in the resource pool.
|
38
|
-
|
39
|
-
### #initialize(limit: number, { block })
|
40
|
-
|
41
|
-
Initializes a new resource pool with the given maximum number of concurrent
|
42
|
-
resources. The given block is used to create the resource.
|
43
|
-
|
44
|
-
```ruby
|
45
|
-
require 'postgres'
|
46
|
-
|
47
|
-
opts = { host: '/tmp', user: 'admin', dbname: 'mydb' }
|
48
|
-
db_connections = Polyphony::ResourcePool.new(limit: 5) { PG.connect(opts) }
|
49
|
-
```
|
50
|
-
|
51
|
-
### #limit → count
|
52
|
-
|
53
|
-
Returns the size limit of the resource pool.
|
54
|
-
|
55
|
-
### #size → count
|
56
|
-
|
57
|
-
Returns the total number of allocated resources in the resource pool. This
|
58
|
-
includes both available and unavailable resources.
|
59
|
-
|
@@ -1,18 +0,0 @@
|
|
1
|
-
---
|
2
|
-
layout: page
|
3
|
-
title: Polyphony::Restart
|
4
|
-
parent: API Reference
|
5
|
-
permalink: /api-reference/polyphony-restart/
|
6
|
-
---
|
7
|
-
# Polyphony::Restart
|
8
|
-
|
9
|
-
`Polyphony::Restart` is an exception class used to restart a fiber. Applications
|
10
|
-
will not normally raise a `Polyphony::Restart` exception, but would rather use
|
11
|
-
`Fiber#restart`.
|
12
|
-
|
13
|
-
```ruby
|
14
|
-
f = spin { do_something_slow }
|
15
|
-
...
|
16
|
-
f.restart
|
17
|
-
...
|
18
|
-
```
|
@@ -1,18 +0,0 @@
|
|
1
|
-
---
|
2
|
-
layout: page
|
3
|
-
title: Polyphony::Terminate
|
4
|
-
parent: API Reference
|
5
|
-
permalink: /api-reference/polyphony-terminate/
|
6
|
-
---
|
7
|
-
# Polyphony::Terminate
|
8
|
-
|
9
|
-
`Polyphony::Terminate` is an exception class used to terminate a fiber without
|
10
|
-
propagating the exception. It should never be rescued. A `Polyphony::Terminate`
|
11
|
-
exception is normally raised using APIs such as `Fiber#terminate` or
|
12
|
-
`Fiber#terminate_all_children`.
|
13
|
-
|
14
|
-
```ruby
|
15
|
-
f = spin { do_something_slow }
|
16
|
-
...
|
17
|
-
f.terminate
|
18
|
-
```
|
@@ -1,67 +0,0 @@
|
|
1
|
-
---
|
2
|
-
layout: page
|
3
|
-
title: Polyphony::ThreadPool
|
4
|
-
parent: API Reference
|
5
|
-
permalink: /api-reference/polyphony-threadpool/
|
6
|
-
---
|
7
|
-
# Polyphony::ThreadPool
|
8
|
-
|
9
|
-
`Polyphony::ThreadPool` implements a general purpose thread pool, normally used
|
10
|
-
for the execution of non-fiber aware operations, such as C-extension based
|
11
|
-
third-party libraries or other system call blocking APIs. The Polyphony
|
12
|
-
implementation of a thread pool allows limiting the number of threads used for
|
13
|
-
performing a recurring operation across one or more fibers.
|
14
|
-
|
15
|
-
A default thread pool is available for quick access to this feature.
|
16
|
-
|
17
|
-
## Class methods
|
18
|
-
|
19
|
-
### #process({ block }) → object
|
20
|
-
|
21
|
-
Runs the given block on the default thread pool. The default pool will be
|
22
|
-
created on the first call to `#process`. This method will block until the
|
23
|
-
operation has completed. The return value is that of the given block. Any
|
24
|
-
uncaught exception will be propagated to the callsite.
|
25
|
-
|
26
|
-
```ruby
|
27
|
-
result = Polyphony::ThreadPool.process { lengthy_op }
|
28
|
-
```
|
29
|
-
|
30
|
-
## Instance methods
|
31
|
-
|
32
|
-
### #busy? → true or false
|
33
|
-
|
34
|
-
Returns true if operations are currently being run on the thread pool.
|
35
|
-
|
36
|
-
### cast({ block }) → pool
|
37
|
-
|
38
|
-
Runs the given block on one of the threads in the pool in a fire-and-forget
|
39
|
-
manner, without waiting for the operation to complete. Using `#cast` to run an
|
40
|
-
operation means there's no way of knowing if the operation has completed or if
|
41
|
-
any exception has been raised, other than inside the block.
|
42
|
-
|
43
|
-
```ruby
|
44
|
-
my_pool.cast { puts 'Hello world' }
|
45
|
-
do_something_else
|
46
|
-
```
|
47
|
-
|
48
|
-
### #initialize(size = Etc.nprocessors)
|
49
|
-
|
50
|
-
Initializes a new instance of `Polyphony::ThreadPool` with the given maximum
|
51
|
-
number of threads. The default size is the number of available processors
|
52
|
-
(number of CPU cores).
|
53
|
-
|
54
|
-
```ruby
|
55
|
-
my_pool = Polyphony::ThreadPool.new(3)
|
56
|
-
```
|
57
|
-
|
58
|
-
### #process({ block }) → object
|
59
|
-
|
60
|
-
Runs the given block on one of the threads in the thread pool and blocks until
|
61
|
-
the operation has completed. The return value is that of the given block. Any
|
62
|
-
uncaught exception will be propagated to the callsite.
|
63
|
-
|
64
|
-
```ruby
|
65
|
-
pool = Polyphony::ThreadPool.new(3)
|
66
|
-
result = pool.process { lengthy_op }
|
67
|
-
```
|
@@ -1,77 +0,0 @@
|
|
1
|
-
---
|
2
|
-
layout: page
|
3
|
-
title: Polyphony::Throttler
|
4
|
-
parent: API Reference
|
5
|
-
permalink: /api-reference/polyphony-throttler/
|
6
|
-
---
|
7
|
-
# Polyphony::Throttler
|
8
|
-
|
9
|
-
`Polyphony::Throttler` implements general purpose operation throttling, or rate
|
10
|
-
limiting. A `Polyphony::Throttler` instance may be used to limit the rate of an
|
11
|
-
arbitrary operation in a single fiber, or across multiple fibers. For example,
|
12
|
-
an HTTP server can limit the number of requests per second across for each
|
13
|
-
client, or for all clients.
|
14
|
-
|
15
|
-
A throttler is invoked using its `#call` method, e.g.:
|
16
|
-
|
17
|
-
```ruby
|
18
|
-
# throttle rate: one per second
|
19
|
-
throttler = Polyphony::Throttler.new(1)
|
20
|
-
|
21
|
-
10.times do |i|
|
22
|
-
spin_loop { throttler.call { p [i, Time.now] } }
|
23
|
-
end
|
24
|
-
```
|
25
|
-
|
26
|
-
If many throttler instances are created over the application's lifetime, they
|
27
|
-
should be stopped using the `#stop` method in order to prevent memory leaks.
|
28
|
-
This is best done using an `ensure` block:
|
29
|
-
|
30
|
-
```ruby
|
31
|
-
def start_server
|
32
|
-
throttler = Polyphony::Throttler.new(1000)
|
33
|
-
MyServer.start do |req|
|
34
|
-
throttler.call { handle_request(req) }
|
35
|
-
end
|
36
|
-
ensure
|
37
|
-
throttler.stop
|
38
|
-
end
|
39
|
-
```
|
40
|
-
|
41
|
-
## Instance methods
|
42
|
-
|
43
|
-
### #initialize(rate)<br>#initialize(interval: interval)<br>#initialize(rate: rate)
|
44
|
-
|
45
|
-
Initializes the throttler with the given rate. The rate can be specified either
|
46
|
-
as a number signifying the maximum rate per second, or as a keyword argument. If
|
47
|
-
the rate is specified using the `interval:` keyword argument, the value given is
|
48
|
-
the minimum interval between consecutive invocations.
|
49
|
-
|
50
|
-
```ruby
|
51
|
-
# These are all equivalent
|
52
|
-
Polyphony::Throttler.new(10)
|
53
|
-
Polyphony::Throttler.new(rate: 10)
|
54
|
-
Polyphony::Throttler.new(interval: 0.1)
|
55
|
-
```
|
56
|
-
|
57
|
-
### #call({ block }) → object
|
58
|
-
|
59
|
-
Invokes the throttler with the given block. This method will sleep for an
|
60
|
-
interval of time required to throttle the execution of the given block. The
|
61
|
-
return value is the return value of the given block.
|
62
|
-
|
63
|
-
### #stop → throttler
|
64
|
-
|
65
|
-
Stops the timer associated with the throttler. This method should be called when
|
66
|
-
the throttler is no longer needed. This is best done from an `ensure` block.
|
67
|
-
|
68
|
-
```ruby
|
69
|
-
def start_server
|
70
|
-
throttler = Polyphony::Throttler.new(1000)
|
71
|
-
MyServer.start do |req|
|
72
|
-
throttler.call { handle_request(req) }
|
73
|
-
end
|
74
|
-
ensure
|
75
|
-
throttler.stop
|
76
|
-
end
|
77
|
-
```
|
@@ -1,36 +0,0 @@
|
|
1
|
-
---
|
2
|
-
layout: page
|
3
|
-
title: Polyphony
|
4
|
-
parent: API Reference
|
5
|
-
permalink: /api-reference/polyphony/
|
6
|
-
---
|
7
|
-
# Polyphony
|
8
|
-
|
9
|
-
The `Polyphony` module acts as a namespace containing general Polyphony
|
10
|
-
functionalities.
|
11
|
-
|
12
|
-
## Class Methods
|
13
|
-
|
14
|
-
### #emit_signal_exception(exception, fiber = Thread.main.main_fiber) → thread
|
15
|
-
|
16
|
-
Emits an exception to the given fiber from a signal handler.
|
17
|
-
|
18
|
-
### #fork({ block }) → pid
|
19
|
-
|
20
|
-
Forks a child process running the given block. Due to the way Ruby implements
|
21
|
-
fibers, along with how signals interact with them, Polyphony-based applications
|
22
|
-
should use `Polyphony#fork` rather than `Kernel#fork`. In order to continue
|
23
|
-
handling fiber scheduling and signal handling correctly, the child process does
|
24
|
-
the following:
|
25
|
-
|
26
|
-
- A new fiber is created using `Fiber#new` and control is transferred to it.
|
27
|
-
- Notify the event loop that a fork has occurred (by calling `ev_loop_fork`).
|
28
|
-
- Setup the current fiber as the main thread's main fiber.
|
29
|
-
- Setup fiber scheduling for the main thread.
|
30
|
-
- Install fiber-aware signal handlers for the `TERM` and `INT` signals.
|
31
|
-
- Run the block.
|
32
|
-
- Correctly handle uncaught exceptions, including `SystemExit` and `Interrupt`.
|
33
|
-
|
34
|
-
### #watch_process(cmd = nil, { block })
|
35
|
-
|
36
|
-
Alternative for [`Polyphony::Process.watch`](../polyphony-process/#watchcmd--nil--block-).
|
@@ -1,88 +0,0 @@
|
|
1
|
-
---
|
2
|
-
layout: page
|
3
|
-
title: ::Thread
|
4
|
-
parent: API Reference
|
5
|
-
permalink: /api-reference/thread/
|
6
|
-
---
|
7
|
-
# ::Thread
|
8
|
-
|
9
|
-
[Ruby core Thread documentation](https://ruby-doc.org/core-2.7.0/Thread.html)
|
10
|
-
|
11
|
-
Polyphony enhances the core `Thread` class with APIs for switching and
|
12
|
-
scheduling fibers, and reimplements some of its APIs such as `Thread#raise`
|
13
|
-
using fibers which, incidentally, make it safe.
|
14
|
-
|
15
|
-
Each thread has its own run queue and its own system backend. While running
|
16
|
-
multiple threads does not result in true parallelism in MRI Ruby, sometimes
|
17
|
-
multithreading is inevitable, for instance when using third-party gems that
|
18
|
-
spawn threads, or when calling blocking APIs that are not fiber-aware.
|
19
|
-
|
20
|
-
## Class Methods
|
21
|
-
|
22
|
-
## Instance methods
|
23
|
-
|
24
|
-
### #<<(object) → fiber<br>#send(object) → fiber
|
25
|
-
|
26
|
-
Sends a message to the thread's main fiber. For further details see
|
27
|
-
[`Fiber#<<`](../fiber/#object--fibersendobject--fiber).
|
28
|
-
|
29
|
-
### #fiber_scheduling_stats → stats
|
30
|
-
|
31
|
-
Returns statistics relating to fiber scheduling for the thread with the
|
32
|
-
following entries:
|
33
|
-
|
34
|
-
- `:scheduled_fibers` - number of fibers currently in the run queue
|
35
|
-
- `:pending_watchers` - number of currently pending event watchers
|
36
|
-
|
37
|
-
### #join → object<br>#await → object
|
38
|
-
|
39
|
-
Waits for the thread to finish running. If the thread has terminated with an
|
40
|
-
uncaught exception, it will be reraised in the context of the calling fiber. If
|
41
|
-
no excecption is raised, returns the thread's result.
|
42
|
-
|
43
|
-
```ruby
|
44
|
-
t = Thread.new { sleep 1 }
|
45
|
-
t.join
|
46
|
-
```
|
47
|
-
|
48
|
-
### #main_fiber → fiber
|
49
|
-
|
50
|
-
Returns the main fiber for the thread.
|
51
|
-
|
52
|
-
### #result → object
|
53
|
-
|
54
|
-
Returns the result of the thread's main fiber.
|
55
|
-
|
56
|
-
```ruby
|
57
|
-
t = Thread.new { 'foo' }
|
58
|
-
t.join
|
59
|
-
t.result #=> 'foo'
|
60
|
-
```
|
61
|
-
|
62
|
-
### #switch_fiber
|
63
|
-
|
64
|
-
invokes a switchpoint, selecting and resuming the next fiber to run. The
|
65
|
-
switching algorithm works as follows:
|
66
|
-
|
67
|
-
- If the run queue is not empty, conditionally run the event loop a single time
|
68
|
-
in order to prevent event starvation when there's always runnable fibers
|
69
|
-
waiting to be resumed.
|
70
|
-
- If the run queue is empty, run the event loop until a fiber is put on the run
|
71
|
-
queue.
|
72
|
-
- Switch to the first fiber in the run queue.
|
73
|
-
|
74
|
-
This method is normally not called directly by the application. Calling
|
75
|
-
`Thread#switch_fiber` means the current fiber has no more work to do and would
|
76
|
-
like yield to other fibers. Note that if the current fiber needs to resume at a
|
77
|
-
later time, it should be scheduled before calling `Thread#switch_fiber`.
|
78
|
-
|
79
|
-
```ruby
|
80
|
-
# schedule current fiber to be resumed later
|
81
|
-
Fiber.current.schedule
|
82
|
-
|
83
|
-
# switch to another fiber
|
84
|
-
Thread.current.switch_fiber
|
85
|
-
|
86
|
-
# the fiber is resumed
|
87
|
-
resume_work
|
88
|
-
```
|
data/docs/favicon.ico
DELETED
Binary file
|
@@ -1,34 +0,0 @@
|
|
1
|
-
---
|
2
|
-
layout: page
|
3
|
-
title: Installing Polyphony
|
4
|
-
parent: Getting Started
|
5
|
-
nav_order: 1
|
6
|
-
---
|
7
|
-
# Installing Polyphony
|
8
|
-
|
9
|
-
## System Requirements
|
10
|
-
|
11
|
-
In order to use Polyphony you need to have:
|
12
|
-
|
13
|
-
- Linux or MacOS (support for Windows will come at a later stage)
|
14
|
-
- Ruby (MRI) 2.6 or newer
|
15
|
-
|
16
|
-
## Installing the Polyphony Gem
|
17
|
-
|
18
|
-
Add this line to your application's Gemfile:
|
19
|
-
|
20
|
-
```ruby
|
21
|
-
gem 'polyphony'
|
22
|
-
```
|
23
|
-
|
24
|
-
And then execute:
|
25
|
-
|
26
|
-
```bash
|
27
|
-
$ bundle
|
28
|
-
```
|
29
|
-
|
30
|
-
Or install it yourself as:
|
31
|
-
|
32
|
-
```bash
|
33
|
-
$ gem install polyphony
|
34
|
-
```
|
File without changes
|
File without changes
|
File without changes
|