polyphony 0.20 → 0.21
Sign up to get free protection for your applications and to get access to all the features.
- checksums.yaml +4 -4
- data/.gitbook.yaml +1 -2
- data/.rubocop.yml +1 -0
- data/CHANGELOG.md +10 -0
- data/Gemfile.lock +1 -1
- data/README.md +18 -449
- data/TODO.md +0 -10
- data/docs/README.md +39 -0
- data/docs/getting-started/installing.md +28 -0
- data/docs/getting-started/tutorial.md +133 -0
- data/docs/summary.md +37 -3
- data/docs/technical-overview/concurrency.md +47 -0
- data/docs/technical-overview/design-principles.md +112 -0
- data/docs/technical-overview/exception-handling.md +34 -41
- data/docs/technical-overview/extending.md +80 -0
- data/docs/technical-overview/faq.md +74 -0
- data/docs/technical-overview/fiber-scheduling.md +23 -52
- data/docs/user-guide/web-server.md +129 -0
- data/examples/core/01-spinning-up-coprocesses.rb +21 -0
- data/examples/core/02-awaiting-coprocesses.rb +18 -0
- data/examples/core/03-interrupting.rb +34 -0
- data/examples/core/04-no-auto-run.rb +18 -0
- data/examples/core/mem-usage.rb +34 -0
- data/examples/core/spin_error.rb +0 -1
- data/examples/core/spin_uncaught_error.rb +0 -1
- data/examples/core/wait_for_signal.rb +14 -0
- data/examples/http/http_server_graceful.rb +25 -0
- data/examples/http/http_server_simple.rb +11 -0
- data/examples/interfaces/redis_pubsub_perf.rb +1 -1
- data/ext/gyro/async.c +4 -40
- data/ext/gyro/child.c +0 -42
- data/ext/gyro/io.c +0 -41
- data/lib/polyphony/core/coprocess.rb +8 -0
- data/lib/polyphony/core/supervisor.rb +29 -10
- data/lib/polyphony/extensions/core.rb +1 -1
- data/lib/polyphony/http/server/http2.rb +20 -4
- data/lib/polyphony/http/server/http2_stream.rb +35 -3
- data/lib/polyphony/version.rb +1 -1
- data/lib/polyphony.rb +17 -5
- data/test/test_async.rb +14 -7
- data/test/test_coprocess.rb +42 -12
- data/test/test_core.rb +26 -0
- data/test/test_io.rb +14 -5
- data/test/test_signal.rb +6 -10
- metadata +17 -5
- data/docs/getting-started/getting-started.md +0 -10
- data/examples/core/spin.rb +0 -14
- data/examples/core/spin_cancel.rb +0 -17
checksums.yaml
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
---
|
2
2
|
SHA256:
|
3
|
-
metadata.gz:
|
4
|
-
data.tar.gz:
|
3
|
+
metadata.gz: 38c4ab294136de3c31b69438b38aaae16f611b87ccd4730cbe4399a9b6e3a7c6
|
4
|
+
data.tar.gz: 35c69e1ff809d340dbe3a121536005338be13c051e8cf2d8c716fd2546c715bc
|
5
5
|
SHA512:
|
6
|
-
metadata.gz:
|
7
|
-
data.tar.gz:
|
6
|
+
metadata.gz: 9d41e1f35a06a5906e21defa0dece49f27faac107764401eceb6e55e9fc6cf68c09ff0fdb109afd8b5b53fee0fdb8ab9ee744dce08490f1c433cc5ec3fe5b7a0
|
7
|
+
data.tar.gz: c69ee7c201fb812fc1c13120884df61f74ba7b4410ffa31a4a70190a8a95e04b2736f8208422c3018373ad02ca025acadadcdc4898b5eaed16071b14b17c00c7
|
data/.gitbook.yaml
CHANGED
data/.rubocop.yml
CHANGED
data/CHANGELOG.md
CHANGED
@@ -1,3 +1,13 @@
|
|
1
|
+
0.21 2019-12-12
|
2
|
+
---------------
|
3
|
+
|
4
|
+
* Add Coprocess.await (for waiting for multiple coprocesses)
|
5
|
+
* Add Coprocess#caller, Coprocess#location methods
|
6
|
+
* Remove callback-oriented Gyro APIs
|
7
|
+
* Revise signal handling API
|
8
|
+
* Improve error handling in HTTP/2 adapter
|
9
|
+
* More documentation
|
10
|
+
|
1
11
|
0.20 2019-11-27
|
2
12
|
---------------
|
3
13
|
|
data/Gemfile.lock
CHANGED
data/README.md
CHANGED
@@ -1,63 +1,34 @@
|
|
1
|
-
# Polyphony -
|
1
|
+
# Polyphony - Easy Concurrency for Ruby
|
2
2
|
|
3
|
-
[
|
4
|
-
[
|
5
|
-
[EXAMPLES](examples) |
|
6
|
-
[TEHNICAL OVERVIEW](#how-polyphony-works---a-technical-overview) |
|
7
|
-
[REFERENCE](#api-reference) |
|
8
|
-
[EXTENDING](#extending-polyphony)
|
3
|
+
[DOCS](https://dfab.gitbook.io/polyphony) |
|
4
|
+
[EXAMPLES](examples)
|
9
5
|
|
10
|
-
> Polyphony
|
11
|
-
> number of parts, each forming an individual melody and harmonizing with each
|
12
|
-
> other.
|
13
|
-
|
14
|
-
**Note**: Polyphony is experimental software. It is designed to work with recent
|
15
|
-
versions of Ruby (2.6 and newer) and supports Linux and MacOS only.
|
6
|
+
> Polyphony \| pəˈlɪf\(ə\)ni \| _Music_ - the style of simultaneously combining a number of parts, each forming an individual melody and harmonizing with each other.
|
16
7
|
|
17
8
|
## What is Polyphony
|
18
9
|
|
19
|
-
Polyphony is a library for building concurrent applications in Ruby. Polyphony
|
20
|
-
harnesses the power of
|
21
|
-
[Ruby fibers](https://ruby-doc.org/core-2.5.1/Fiber.html) to provide a
|
22
|
-
cooperative, sequential coprocess-based concurrency model. Under the hood,
|
23
|
-
Polyphony uses [libev](https://github.com/enki/libev) as a high-performance event
|
24
|
-
reactor that provides timers, I/O watchers and other asynchronous event
|
25
|
-
primitives.
|
10
|
+
Polyphony is a library for building concurrent applications in Ruby. Polyphony harnesses the power of [Ruby fibers](https://ruby-doc.org/core-2.5.1/Fiber.html) to provide a cooperative, sequential coprocess-based concurrency model. Under the hood, Polyphony uses [libev](https://github.com/enki/libev) as a high-performance event reactor that provides timers, I/O watchers and other asynchronous event primitives.
|
26
11
|
|
27
|
-
Polyphony makes it possible to use normal Ruby built-in classes like `IO`, and
|
28
|
-
`Socket` in a concurrent fashion without having to resort to threads. Polyphony
|
29
|
-
takes care of context-switching automatically whenever a blocking call like
|
30
|
-
`Socket#accept` or `IO#read` is issued.
|
12
|
+
Polyphony makes it possible to use normal Ruby built-in classes like `IO`, and `Socket` in a concurrent fashion without having to resort to threads. Polyphony takes care of context-switching automatically whenever a blocking call like `Socket#accept` or `IO#read` is issued.
|
31
13
|
|
32
14
|
## Features
|
33
15
|
|
34
|
-
|
16
|
+
* **Full-blown, integrated, high-performance HTTP 1 / HTTP 2 / WebSocket server
|
35
17
|
with TLS/SSL termination, automatic ALPN protocol selection, and body
|
36
18
|
streaming**.
|
37
|
-
|
38
|
-
|
39
|
-
|
19
|
+
* Co-operative scheduling of concurrent tasks using Ruby fibers.
|
20
|
+
* High-performance event reactor for handling I/O events and timers.
|
21
|
+
* Natural, sequential programming style that makes it easy to reason about
|
40
22
|
concurrent code.
|
41
|
-
|
23
|
+
* Abstractions and constructs for controlling the execution of concurrent code:
|
42
24
|
coprocesses, supervisors, cancel scopes, throttling, resource pools etc.
|
43
|
-
|
25
|
+
* Code can use native networking classes and libraries, growing support for
|
44
26
|
third-party gems such as `pg` and `redis`.
|
45
|
-
|
46
|
-
|
47
|
-
|
27
|
+
* Use stdlib classes such as `TCPServer`, `TCPSocket` and
|
28
|
+
* HTTP 1 / HTTP 2 client agent with persistent connections.
|
29
|
+
* Competitive performance and scalability characteristics, in terms of both
|
48
30
|
throughput and memory consumption.
|
49
31
|
|
50
|
-
## Why you should not use Polyphony
|
51
|
-
|
52
|
-
- Polyphony does weird things to Ruby, like patching methods like `IO.read`,
|
53
|
-
`Kernel#sleep`, and `Timeout.timeout` so they'll work concurrently without
|
54
|
-
using threads.
|
55
|
-
- Error backtraces might look weird.
|
56
|
-
- There's currently no support for threads - any IO operations in threads will
|
57
|
-
likely cause a bad crash.
|
58
|
-
- Debugging might be confusing or not work at all.
|
59
|
-
- The API is currently unstable.
|
60
|
-
|
61
32
|
## Prior Art
|
62
33
|
|
63
34
|
Polyphony draws inspiration from the following, in no particular order:
|
@@ -70,409 +41,7 @@ Polyphony draws inspiration from the following, in no particular order:
|
|
70
41
|
* [Erlang supervisors](http://erlang.org/doc/man/supervisor.html) (and actually,
|
71
42
|
Erlang in general)
|
72
43
|
|
73
|
-
##
|
74
|
-
|
75
|
-
```bash
|
76
|
-
$ gem install polyphony
|
77
|
-
```
|
78
|
-
|
79
|
-
Or add it to your Gemfile, you know the drill.
|
80
|
-
|
81
|
-
## Getting Started
|
82
|
-
|
83
|
-
Polyphony is designed to help you write high-performance, concurrent code in
|
84
|
-
Ruby, without using threads. It does so by turning every call which might block,
|
85
|
-
such as `Kernel#sleep` or `IO#read` into a concurrent operation, which yields
|
86
|
-
control to an event reactor. The reactor, in turn, may schedule other operations
|
87
|
-
once they can be resumed. In that manner, multiple ongoing operations may be
|
88
|
-
processed concurrently.
|
89
|
-
|
90
|
-
The simplest way to start a concurrent operation is using `Kernel#spin`:
|
91
|
-
|
92
|
-
```ruby
|
93
|
-
require 'polyphony'
|
94
|
-
|
95
|
-
spin do
|
96
|
-
puts "A going to sleep"
|
97
|
-
sleep 1
|
98
|
-
puts "A woken up"
|
99
|
-
end
|
100
|
-
|
101
|
-
spin do
|
102
|
-
puts "B going to sleep"
|
103
|
-
sleep 1
|
104
|
-
puts "B woken up"
|
105
|
-
end
|
106
|
-
```
|
107
|
-
|
108
|
-
In the above example, both `sleep` calls will be executed concurrently, and thus
|
109
|
-
the program will take approximately only 1 second to execute. Note how the logic
|
110
|
-
flow inside each `spin` block is purely sequential, and how the concurrent
|
111
|
-
nature of the two blocks is expressed simply and cleanly.
|
112
|
-
|
113
|
-
## Coprocesses - Polyphony's basic unit of concurrency
|
114
|
-
|
115
|
-
In Polyphony, concurrent operations take place inside coprocesses. A `Coprocess`
|
116
|
-
is executed on top of a `Fiber`, which allows it to be suspended whenever a
|
117
|
-
blocking operation is called, and resumed once that operation has been
|
118
|
-
completed. Coprocesses offer significant advantages over threads - they consume
|
119
|
-
only about 10KB, switching between them is much faster than switching threads,
|
120
|
-
and literally millions of them can be spinned off without affecting
|
121
|
-
performance*. Besides, Ruby does not yet allow parallel execution of threads
|
122
|
-
(courtesy of the Ruby GVL).
|
123
|
-
|
124
|
-
\* *This is a totally unsubstantiated claim and has not been proven in practice*.
|
125
|
-
|
126
|
-
## An echo server in Polyphony
|
127
|
-
|
128
|
-
Let's now examine how networking is done using Polyphony. Here's a bare-bones
|
129
|
-
echo server written using Polyphony:
|
130
|
-
|
131
|
-
```ruby
|
132
|
-
require 'polyphony'
|
133
|
-
|
134
|
-
server = TCPServer.open(1234)
|
135
|
-
while client = server.accept
|
136
|
-
spin do
|
137
|
-
while (data = client.gets)
|
138
|
-
client << data
|
139
|
-
end
|
140
|
-
end
|
141
|
-
end
|
142
|
-
```
|
143
|
-
|
144
|
-
This example demonstrates several features of Polyphony:
|
145
|
-
|
146
|
-
- The code uses the native `TCPServer` class from Ruby's stdlib, to setup a TCP
|
147
|
-
server. The result of `server.accept` is also a native `TCPSocket` object.
|
148
|
-
There are no wrapper classes being used.
|
149
|
-
- The only hint of the code being concurrent is the use of `Kernel#spin`,
|
150
|
-
which starts a new coprocess on a dedicated fiber. This allows serving
|
151
|
-
multiple clients at once. Whenever a blocking call is issued, such as
|
152
|
-
`#accept` or `#read`, execution is *yielded* to the event reactor loop, which
|
153
|
-
will resume only those coprocesses which are ready to be resumed.
|
154
|
-
- Exception handling is done using the normal Ruby constructs `raise`, `rescue`
|
155
|
-
and `ensure`. Exceptions never go unhandled (as might be the case with Ruby
|
156
|
-
threads), and must be dealt with explicitly. An unhandled exception will by
|
157
|
-
default cause the Ruby process to exit.
|
158
|
-
|
159
|
-
## Additional concurrency constructs
|
160
|
-
|
161
|
-
In order to facilitate writing concurrent code, Polyphony provides additional
|
162
|
-
mechanisms that make it easier to create and control concurrent tasks.
|
163
|
-
|
164
|
-
### Cancel scopes
|
165
|
-
|
166
|
-
Cancel scopes, an idea borrowed from Python's
|
167
|
-
[Trio](https://trio.readthedocs.io/) library, are used to cancel the execution
|
168
|
-
of one or more coprocesses. The most common use of cancel scopes is a for
|
169
|
-
implementing a timeout for the completion of a task. Any blocking operation can
|
170
|
-
be cancelled. The programmer may choose to raise a `Cancel` exception when an
|
171
|
-
operation has been cancelled, or alternatively to move on without any exception.
|
172
|
-
|
173
|
-
Cancel scopes are typically started using `Kernel#cancel_after` and
|
174
|
-
`Kernel#move_on_after` for cancelling with or without an exception,
|
175
|
-
respectively. Cancel scopes will take a block of code to execute and run it,
|
176
|
-
providing a reference to the cancel scope:
|
177
|
-
|
178
|
-
```ruby
|
179
|
-
puts "going to sleep (but really only for 1 second)..."
|
180
|
-
cancel_after(1) do
|
181
|
-
sleep(60)
|
182
|
-
end
|
183
|
-
```
|
184
|
-
|
185
|
-
Patterns like closing a connection after X seconds of activity are greatly
|
186
|
-
facilitated by timeout-based cancel scopes, which can be easily reset:
|
187
|
-
|
188
|
-
```ruby
|
189
|
-
def echoer(client)
|
190
|
-
# close connection after 10 seconds of inactivity
|
191
|
-
move_on_after(10) do |scope|
|
192
|
-
scope.when_cancelled { puts "closing connection due to inactivity" }
|
193
|
-
loop do
|
194
|
-
data = client.read
|
195
|
-
scope.reset_timeout
|
196
|
-
client.write
|
197
|
-
end
|
198
|
-
end
|
199
|
-
client.close
|
200
|
-
end
|
201
|
-
```
|
202
|
-
|
203
|
-
Cancel scopes may also be manually cancelled by calling `CancelScope#cancel!`
|
204
|
-
at any time:
|
205
|
-
|
206
|
-
```ruby
|
207
|
-
def echoer(client)
|
208
|
-
move_on_after(60) do |scope|
|
209
|
-
loop do
|
210
|
-
data = client.read
|
211
|
-
scope.cancel! if data == 'stop'
|
212
|
-
client.write
|
213
|
-
end
|
214
|
-
end
|
215
|
-
client.close
|
216
|
-
end
|
217
|
-
```
|
218
|
-
|
219
|
-
### Resource pools
|
220
|
-
|
221
|
-
A resource pool is used to control access to one or more shared, usually
|
222
|
-
identical resources. For example, a resource pool can be used to control
|
223
|
-
concurrent access to database connections, or to limit concurrent
|
224
|
-
requests to an external API:
|
225
|
-
|
226
|
-
```ruby
|
227
|
-
# up to 5 concurrent connections
|
228
|
-
Pool = Polyphony::ResourcePool.new(limit: 5) {
|
229
|
-
# the block sets up the resource
|
230
|
-
PG.connect(...)
|
231
|
-
}
|
232
|
-
|
233
|
-
1000.times {
|
234
|
-
spin {
|
235
|
-
Pool.acquire { |db| p db.query('select 1') }
|
236
|
-
}
|
237
|
-
}
|
238
|
-
```
|
239
|
-
|
240
|
-
You can also call arbitrary methods on the resource pool, which will be
|
241
|
-
delegated to the resource using `#method_missing`:
|
242
|
-
|
243
|
-
```ruby
|
244
|
-
# up to 5 concurrent connections
|
245
|
-
Pool = Polyphony::ResourcePool.new(limit: 5) {
|
246
|
-
# the block sets up the resource
|
247
|
-
PG.connect(...)
|
248
|
-
}
|
249
|
-
|
250
|
-
1000.times {
|
251
|
-
spin { p Pool.query('select pg_sleep(0.01);') }
|
252
|
-
}
|
253
|
-
```
|
254
|
-
|
255
|
-
### Supervisors
|
256
|
-
|
257
|
-
A supervisor is used to control one or more coprocesses. It can be used to
|
258
|
-
start, stop, restart and await the completion of multiple coprocesses. It is
|
259
|
-
normally started using `Kernel#supervise`:
|
260
|
-
|
261
|
-
```ruby
|
262
|
-
supervise { |s|
|
263
|
-
s.spin { sleep 1 }
|
264
|
-
s.spin { sleep 2 }
|
265
|
-
s.spin { sleep 3 }
|
266
|
-
}
|
267
|
-
puts "done sleeping"
|
268
|
-
```
|
269
|
-
|
270
|
-
The `Kernel#supervise` method will await the completion of all supervised
|
271
|
-
coprocesses. If any supervised coprocess raises an error, the supervisor will
|
272
|
-
automatically cancel all other supervised coprocesses.
|
273
|
-
|
274
|
-
### Throttlers
|
275
|
-
|
276
|
-
A throttler is a mechanism for controlling the speed of an arbitrary task,
|
277
|
-
such as sending of emails, or crawling a website. A throttler is normally
|
278
|
-
created using `Kernel#throttle` or `Kernel#throttled_loop`, and can even be used
|
279
|
-
to throttle operations across multiple coprocesses:
|
280
|
-
|
281
|
-
```ruby
|
282
|
-
server = Polyphony::Net.tcp_listen(1234)
|
283
|
-
|
284
|
-
# a shared throttler, up to 10 times per second
|
285
|
-
throttler = throttle(rate: 10)
|
286
|
-
|
287
|
-
while client = server.accept
|
288
|
-
spin do
|
289
|
-
throttler.call do
|
290
|
-
while data = client.read
|
291
|
-
client.write(data)
|
292
|
-
end
|
293
|
-
end
|
294
|
-
end
|
295
|
-
end
|
296
|
-
```
|
297
|
-
|
298
|
-
`Kernel#throttled_loop` can be used to run throttled infinite loops:
|
299
|
-
|
300
|
-
```ruby
|
301
|
-
throttled_loop(3) do
|
302
|
-
STDOUT << '.'
|
303
|
-
end
|
304
|
-
```
|
305
|
-
|
306
|
-
## Going further
|
307
|
-
|
308
|
-
To learn more about using Polyphony to build concurrent applications, read the
|
309
|
-
technical overview below, or look at the [included examples](examples). A
|
310
|
-
thorough reference is forthcoming.
|
311
|
-
|
312
|
-
## How Polyphony Works - a Technical Overview
|
313
|
-
|
314
|
-
### Fiber-based concurrency
|
315
|
-
|
316
|
-
The built-in `Fiber` class provides a very elegant, if low-level, foundation for
|
317
|
-
implementing cooperative, light-weight concurrency (it can also be used for other stuff like generators). Fiber or continuation-based concurrency can be
|
318
|
-
considered as the
|
319
|
-
[*third way*](https://en.wikipedia.org/wiki/Fiber_(computer_science))
|
320
|
-
of writing concurrent programs (the other two being multi-process concurrency
|
321
|
-
and multi-thread concurrency), and can provide very good performance
|
322
|
-
characteristics for I/O-bound applications.
|
323
|
-
|
324
|
-
In contrast to callback-based concurrency (e.g. Node.js or EventMachine), fibers
|
325
|
-
allow writing concurrent code in a sequential manner without having to split
|
326
|
-
your logic into different locations, or submitting to
|
327
|
-
[callback hell](http://callbackhell.com/).
|
328
|
-
|
329
|
-
Polyphony builds on the foundation of Ruby fibers in order to facilitate writing
|
330
|
-
high-performance I/O-bound applications in Ruby.
|
331
|
-
|
332
|
-
### Context-switching on blocking calls
|
333
|
-
|
334
|
-
Ruby monkey-patches existing methods such as `sleep` or `IO#read` to setup an
|
335
|
-
IO watcher and suspend the current fiber until the IO object is ready to be
|
336
|
-
read. Once the IO watcher is signalled, the associated fiber is resumed and the
|
337
|
-
method call can continue. Here's a simplified implementation of
|
338
|
-
[`IO#read`](lib/polyphony/io.rb#24-36):
|
339
|
-
|
340
|
-
```ruby
|
341
|
-
class IO
|
342
|
-
def read(max = 8192)
|
343
|
-
loop do
|
344
|
-
result = read_nonblock(max, exception: false)
|
345
|
-
case result
|
346
|
-
when nil then raise IOError
|
347
|
-
when :wait_readable then read_watcher.await
|
348
|
-
else return result
|
349
|
-
end
|
350
|
-
end
|
351
|
-
end
|
352
|
-
end
|
353
|
-
```
|
354
|
-
|
355
|
-
The magic starts in [`IOWatcher#await`](ext/ev/io.c#157-179), where the watcher
|
356
|
-
is started and the current fiber is suspended (it "yields" in Ruby parlance).
|
357
|
-
Here's a naïve implementation (the actual implementation is written in C):
|
358
|
-
|
359
|
-
```ruby
|
360
|
-
class IOWatcher
|
361
|
-
def await
|
362
|
-
@fiber = Fiber.current
|
363
|
-
start
|
364
|
-
yield_to_reactor_fiber
|
365
|
-
end
|
366
|
-
end
|
367
|
-
```
|
368
|
-
|
369
|
-
> **Running a high-performance event loop**: Polyphony runs a libev-based event
|
370
|
-
> loop that watches events such as IO-readiness, elapsed timers, received
|
371
|
-
> signals and other asynchronous happenings, and uses them to control fiber
|
372
|
-
> execution. The event loop itself is run on a separate fiber, allowing the main
|
373
|
-
> fiber as well to perform blocking operations.
|
374
|
-
|
375
|
-
When the IO watcher is [signalled](ext/ev/io.c#99-116): the fiber associated
|
376
|
-
with the watcher is resumed, and control is given back to the calling method.
|
377
|
-
Here's a naïve implementation:
|
378
|
-
|
379
|
-
```ruby
|
380
|
-
class IOWatcher
|
381
|
-
def signal
|
382
|
-
@fiber.transfer
|
383
|
-
end
|
384
|
-
end
|
385
|
-
```
|
386
|
-
|
387
|
-
## API Reference
|
388
|
-
|
389
|
-
To be continued...
|
390
|
-
|
391
|
-
## Extending Polyphony
|
392
|
-
|
393
|
-
Polyphony was designed to ease the transition from blocking APIs and
|
394
|
-
callback-based API to non-blocking, fiber-based ones. It is important to
|
395
|
-
understand that not all blocking calls can be easily converted into
|
396
|
-
non-blocking calls. That might be the case with Ruby gems based on C-extensions,
|
397
|
-
such as database libraries. In that case, Polyphony's built-in
|
398
|
-
[thread pool](#threadpool) might be used for offloading such blocking calls.
|
399
|
-
|
400
|
-
### Adapting callback-based APIs
|
401
|
-
|
402
|
-
Some of the most common patterns in Ruby APIs is the callback pattern, in which
|
403
|
-
the API takes a block as a callback to be called upon completion of a task. One
|
404
|
-
such example can be found in the excellent
|
405
|
-
[http_parser.rb](https://github.com/tmm1/http_parser.rb/) gem, which is used by
|
406
|
-
Polyphony itself to provide HTTP 1 functionality. The `HTTP:Parser` provides
|
407
|
-
multiple hooks, or callbacks, for being notified when an HTTP request is
|
408
|
-
complete. The typical callback-based setup is as follows:
|
409
|
-
|
410
|
-
```ruby
|
411
|
-
require 'http/parser'
|
412
|
-
@parser = Http::Parser.new
|
413
|
-
|
414
|
-
def on_receive(data)
|
415
|
-
@parser < data
|
416
|
-
end
|
417
|
-
|
418
|
-
@parser.on_message_complete do |env|
|
419
|
-
process_request(env)
|
420
|
-
end
|
421
|
-
```
|
422
|
-
|
423
|
-
A program using `http_parser.rb` in conjunction with Polyphony might do the
|
424
|
-
following:
|
425
|
-
|
426
|
-
```ruby
|
427
|
-
require 'http/parser'
|
428
|
-
require 'polyphony'
|
429
|
-
|
430
|
-
def handle_client(client)
|
431
|
-
parser = Http::Parser.new
|
432
|
-
req = nil
|
433
|
-
parser.on_message_complete { |env| req = env }
|
434
|
-
loop do
|
435
|
-
parser << client.read
|
436
|
-
if req
|
437
|
-
handle_request(req)
|
438
|
-
req = nil
|
439
|
-
end
|
440
|
-
end
|
441
|
-
end
|
442
|
-
```
|
443
|
-
|
444
|
-
Another possibility would be to monkey-patch `Http::Parser` in order to
|
445
|
-
encapsulate the state of the request:
|
446
|
-
|
447
|
-
```ruby
|
448
|
-
class Http::Parser
|
449
|
-
def setup
|
450
|
-
self.on_message_complete = proc { @request_complete = true }
|
451
|
-
end
|
452
|
-
|
453
|
-
def parser(data)
|
454
|
-
self << data
|
455
|
-
return nil unless @request_complete
|
456
|
-
|
457
|
-
@request_complete = nil
|
458
|
-
self
|
459
|
-
end
|
460
|
-
end
|
461
|
-
|
462
|
-
def handle_client(client)
|
463
|
-
parser = Http::Parser.new
|
464
|
-
loop do
|
465
|
-
if req == parser.parse(client.read)
|
466
|
-
handle_request(req)
|
467
|
-
end
|
468
|
-
end
|
469
|
-
end
|
470
|
-
```
|
471
|
-
|
472
|
-
### Contributing to Polyphony
|
44
|
+
## Documentation
|
473
45
|
|
474
|
-
|
475
|
-
|
476
|
-
[creating an issue](https://github.com/digital-fabric/polyphony/issues). Our aim
|
477
|
-
is for Polyphony to be a comprehensive solution for writing concurrent Ruby
|
478
|
-
programs.
|
46
|
+
The complete documentation for Polyphony could be found on the
|
47
|
+
[Polyphony website](https://dfab.gitbook.io/polyphony).
|
data/TODO.md
CHANGED
@@ -6,16 +6,6 @@
|
|
6
6
|
- Work better mechanism supervising multiple coprocesses (`when_done` feels a
|
7
7
|
bit hacky)
|
8
8
|
|
9
|
-
## 0.21 REPL usage, coprocess introspection, monitoring
|
10
|
-
|
11
|
-
- Implement `move_on_after(1, with: nil) { ... }`
|
12
|
-
- Implement `Coprocess.await` for waiting on multiple coprocesses without
|
13
|
-
starting them in a supervisor, will also necessitate adding `Supervisor#add`
|
14
|
-
- Implement `Coprocess#location`
|
15
|
-
- Implement `Coprocess#alive?`
|
16
|
-
- Implement `Coprocess#caller` - points to coprocess that called the coprocess
|
17
|
-
- Implement `Coprocess.list` - a list of running coprocesses
|
18
|
-
|
19
9
|
## 0.22 Full Rack adapter implementation
|
20
10
|
|
21
11
|
- Homogenize HTTP 1 and HTTP 2 headers - upcase ? downcase ?
|
data/docs/README.md
ADDED
@@ -0,0 +1,39 @@
|
|
1
|
+
# Polyphony - Easy Concurrency for Ruby
|
2
|
+
|
3
|
+
> Polyphony \| pəˈlɪf\(ə\)ni \| _Music_ - the style of simultaneously combining a number of parts, each forming an individual melody and harmonizing with each other.
|
4
|
+
|
5
|
+
## What is Polyphony
|
6
|
+
|
7
|
+
Polyphony is a library for building concurrent applications in Ruby. Polyphony harnesses the power of [Ruby fibers](https://ruby-doc.org/core-2.5.1/Fiber.html) to provide a cooperative, sequential coprocess-based concurrency model. Under the hood, Polyphony uses [libev](https://github.com/enki/libev) as a high-performance event reactor that provides timers, I/O watchers and other asynchronous event primitives.
|
8
|
+
|
9
|
+
Polyphony makes it possible to use normal Ruby built-in classes like `IO`, and `Socket` in a concurrent fashion without having to resort to threads. Polyphony takes care of context-switching automatically whenever a blocking call like `Socket#accept` or `IO#read` is issued.
|
10
|
+
|
11
|
+
## Features
|
12
|
+
|
13
|
+
* **Full-blown, integrated, high-performance HTTP 1 / HTTP 2 / WebSocket server with TLS/SSL termination, automatic ALPN protocol selection, and body streaming**.
|
14
|
+
* Co-operative scheduling of concurrent tasks using Ruby fibers.
|
15
|
+
* High-performance event reactor for handling I/O events and timers.
|
16
|
+
* Natural, sequential programming style that makes it easy to reason about concurrent code.
|
17
|
+
* Abstractions and constructs for controlling the execution of concurrent code: coprocesses, supervisors, cancel scopes, throttling, resource pools etc.
|
18
|
+
* Code can use native networking classes and libraries, growing support for third-party gems such as `pg` and `redis`.
|
19
|
+
* Use stdlib classes such as `TCPServer` and `TCPSocket` and `Net::HTTP`.
|
20
|
+
* HTTP 1 / HTTP 2 client agent with persistent connections.
|
21
|
+
* Competitive performance and scalability characteristics, in terms of both throughput and memory consumption.
|
22
|
+
|
23
|
+
## Prior Art
|
24
|
+
|
25
|
+
Polyphony draws inspiration from the following, in no particular order:
|
26
|
+
|
27
|
+
* [nio4r](https://github.com/socketry/nio4r/) and [async](https://github.com/socketry/async) (Polyphony's C-extension code is largely a spinoff of [nio4r's](https://github.com/socketry/nio4r/tree/master/ext))
|
28
|
+
* [EventMachine](https://github.com/eventmachine/eventmachine)
|
29
|
+
* [Trio](https://trio.readthedocs.io/)
|
30
|
+
* [Erlang supervisors](http://erlang.org/doc/man/supervisor.html) (and actually, Erlang in general)
|
31
|
+
|
32
|
+
## Going further
|
33
|
+
|
34
|
+
To learn more about using Polyphony to build concurrent applications, read the technical overview below, or look at the [included examples](https://github.com/digital-fabric/polyphony/tree/9e0f3b09213156bdf376ef33684ef267517f06e8/examples/README.md). A thorough reference is forthcoming.
|
35
|
+
|
36
|
+
## Contributing to Polyphony
|
37
|
+
|
38
|
+
If there's some blocking behavior you'd like to see handled by Polyphony, please let us know by [creating an issue](https://github.com/digital-fabric/polyphony/issues). Our aim is for Polyphony to be a comprehensive solution for writing concurrent Ruby programs.
|
39
|
+
|
@@ -0,0 +1,28 @@
|
|
1
|
+
## System Requirements
|
2
|
+
|
3
|
+
In order to use Polyphony you need to have:
|
4
|
+
|
5
|
+
- Linux or MacOS (sorry, there are no plans to support Windows for the time
|
6
|
+
being)
|
7
|
+
- Ruby (MRI) 2.6 or newer
|
8
|
+
|
9
|
+
## Installing Polyphony
|
10
|
+
|
11
|
+
Add this line to your application's Gemfile:
|
12
|
+
|
13
|
+
```ruby
|
14
|
+
gem 'polyphony'
|
15
|
+
```
|
16
|
+
|
17
|
+
And then execute:
|
18
|
+
|
19
|
+
```bash
|
20
|
+
$ bundle
|
21
|
+
```
|
22
|
+
|
23
|
+
Or install it yourself as:
|
24
|
+
|
25
|
+
```bash
|
26
|
+
$ gem install polyphony
|
27
|
+
```
|
28
|
+
|