polyphony 0.27 → 0.28

Sign up to get free protection for your applications and to get access to all the features.
Files changed (63) hide show
  1. checksums.yaml +4 -4
  2. data/.gitignore +4 -1
  3. data/CHANGELOG.md +13 -0
  4. data/Gemfile +12 -1
  5. data/Gemfile.lock +83 -5
  6. data/Rakefile +4 -0
  7. data/TODO.md +11 -20
  8. data/docs/_config.yml +15 -0
  9. data/docs/_includes/nav.html +47 -0
  10. data/docs/_sass/custom/custom.scss +5 -0
  11. data/docs/_sass/overrides.scss +45 -0
  12. data/docs/assets/img/echo-fibers.svg +1 -0
  13. data/docs/assets/img/sleeping-fiber.svg +1 -0
  14. data/docs/faq.md +182 -0
  15. data/docs/getting-started/installing.md +10 -2
  16. data/docs/getting-started/tutorial.md +333 -26
  17. data/docs/getting-started.md +10 -0
  18. data/docs/index.md +91 -0
  19. data/docs/technical-overview/concurrency.md +78 -16
  20. data/docs/technical-overview/design-principles.md +7 -0
  21. data/docs/technical-overview/exception-handling.md +57 -9
  22. data/docs/technical-overview/extending.md +7 -0
  23. data/docs/technical-overview/fiber-scheduling.md +128 -18
  24. data/docs/technical-overview.md +10 -0
  25. data/docs/user-guide/web-server.md +7 -0
  26. data/docs/user-guide.md +10 -0
  27. data/examples/core/xx-deadlock.rb +8 -0
  28. data/examples/core/xx-state-machine.rb +51 -0
  29. data/examples/core/xx-trace.rb +80 -0
  30. data/examples/interfaces/pg_notify.rb +35 -0
  31. data/examples/io/xx-httparty.rb +31 -6
  32. data/examples/io/xx-irb.rb +1 -11
  33. data/examples/io/xx-switch.rb +15 -0
  34. data/ext/gyro/gyro.c +77 -38
  35. data/ext/gyro/gyro.h +15 -5
  36. data/ext/gyro/gyro_ext.c +3 -0
  37. data/ext/gyro/thread.c +47 -32
  38. data/ext/gyro/tracing.c +11 -0
  39. data/lib/polyphony/core/global_api.rb +11 -4
  40. data/lib/polyphony/core/supervisor.rb +1 -0
  41. data/lib/polyphony/core/thread_pool.rb +44 -35
  42. data/lib/polyphony/extensions/fiber.rb +19 -9
  43. data/lib/polyphony/extensions/io.rb +14 -14
  44. data/lib/polyphony/extensions/socket.rb +3 -3
  45. data/lib/polyphony/irb.rb +13 -0
  46. data/lib/polyphony/postgres.rb +15 -0
  47. data/lib/polyphony/trace.rb +98 -0
  48. data/lib/polyphony/version.rb +1 -1
  49. data/lib/polyphony.rb +1 -0
  50. data/polyphony.gemspec +21 -12
  51. data/test/helper.rb +3 -2
  52. data/test/test_fiber.rb +53 -3
  53. data/test/test_global_api.rb +12 -0
  54. data/test/test_gyro.rb +2 -2
  55. data/test/test_supervisor.rb +12 -0
  56. data/test/test_thread.rb +12 -0
  57. data/test/test_thread_pool.rb +75 -0
  58. data/test/test_throttler.rb +6 -0
  59. data/test/test_trace.rb +66 -0
  60. metadata +99 -9
  61. data/docs/README.md +0 -36
  62. data/docs/summary.md +0 -60
  63. data/docs/technical-overview/faq.md +0 -97
metadata CHANGED
@@ -1,14 +1,14 @@
1
1
  --- !ruby/object:Gem::Specification
2
2
  name: polyphony
3
3
  version: !ruby/object:Gem::Version
4
- version: '0.27'
4
+ version: '0.28'
5
5
  platform: ruby
6
6
  authors:
7
7
  - Sharon Rosner
8
8
  autorequire:
9
9
  bindir: bin
10
10
  cert_chain: []
11
- date: 2020-01-19 00:00:00.000000000 Z
11
+ date: 2020-01-27 00:00:00.000000000 Z
12
12
  dependencies:
13
13
  - !ruby/object:Gem::Dependency
14
14
  name: modulation
@@ -58,14 +58,14 @@ dependencies:
58
58
  requirements:
59
59
  - - '='
60
60
  - !ruby/object:Gem::Version
61
- version: 5.11.3
61
+ version: 5.13.0
62
62
  type: :development
63
63
  prerelease: false
64
64
  version_requirements: !ruby/object:Gem::Requirement
65
65
  requirements:
66
66
  - - '='
67
67
  - !ruby/object:Gem::Version
68
- version: 5.11.3
68
+ version: 5.13.0
69
69
  - !ruby/object:Gem::Dependency
70
70
  name: minitest-reporters
71
71
  requirement: !ruby/object:Gem::Requirement
@@ -94,6 +94,20 @@ dependencies:
94
94
  - - '='
95
95
  - !ruby/object:Gem::Version
96
96
  version: 0.17.1
97
+ - !ruby/object:Gem::Dependency
98
+ name: rubocop
99
+ requirement: !ruby/object:Gem::Requirement
100
+ requirements:
101
+ - - '='
102
+ - !ruby/object:Gem::Version
103
+ version: 0.79.0
104
+ type: :development
105
+ prerelease: false
106
+ version_requirements: !ruby/object:Gem::Requirement
107
+ requirements:
108
+ - - '='
109
+ - !ruby/object:Gem::Version
110
+ version: 0.79.0
97
111
  - !ruby/object:Gem::Dependency
98
112
  name: pg
99
113
  requirement: !ruby/object:Gem::Requirement
@@ -164,6 +178,62 @@ dependencies:
164
178
  - - "~>"
165
179
  - !ruby/object:Gem::Version
166
180
  version: 0.6.0
181
+ - !ruby/object:Gem::Dependency
182
+ name: jekyll
183
+ requirement: !ruby/object:Gem::Requirement
184
+ requirements:
185
+ - - "~>"
186
+ - !ruby/object:Gem::Version
187
+ version: 3.8.6
188
+ type: :development
189
+ prerelease: false
190
+ version_requirements: !ruby/object:Gem::Requirement
191
+ requirements:
192
+ - - "~>"
193
+ - !ruby/object:Gem::Version
194
+ version: 3.8.6
195
+ - !ruby/object:Gem::Dependency
196
+ name: jekyll-remote-theme
197
+ requirement: !ruby/object:Gem::Requirement
198
+ requirements:
199
+ - - "~>"
200
+ - !ruby/object:Gem::Version
201
+ version: 0.4.1
202
+ type: :development
203
+ prerelease: false
204
+ version_requirements: !ruby/object:Gem::Requirement
205
+ requirements:
206
+ - - "~>"
207
+ - !ruby/object:Gem::Version
208
+ version: 0.4.1
209
+ - !ruby/object:Gem::Dependency
210
+ name: jekyll-seo-tag
211
+ requirement: !ruby/object:Gem::Requirement
212
+ requirements:
213
+ - - "~>"
214
+ - !ruby/object:Gem::Version
215
+ version: 2.6.1
216
+ type: :development
217
+ prerelease: false
218
+ version_requirements: !ruby/object:Gem::Requirement
219
+ requirements:
220
+ - - "~>"
221
+ - !ruby/object:Gem::Version
222
+ version: 2.6.1
223
+ - !ruby/object:Gem::Dependency
224
+ name: just-the-docs
225
+ requirement: !ruby/object:Gem::Requirement
226
+ requirements:
227
+ - - "~>"
228
+ - !ruby/object:Gem::Version
229
+ version: 0.2.7
230
+ type: :development
231
+ prerelease: false
232
+ version_requirements: !ruby/object:Gem::Requirement
233
+ requirements:
234
+ - - "~>"
235
+ - !ruby/object:Gem::Version
236
+ version: 0.2.7
167
237
  description:
168
238
  email: ciconia@gmail.com
169
239
  executables: []
@@ -183,21 +253,30 @@ files:
183
253
  - README.md
184
254
  - Rakefile
185
255
  - TODO.md
186
- - docs/README.md
256
+ - docs/_config.yml
257
+ - docs/_includes/nav.html
258
+ - docs/_sass/custom/custom.scss
259
+ - docs/_sass/overrides.scss
260
+ - docs/assets/img/echo-fibers.svg
261
+ - docs/assets/img/sleeping-fiber.svg
262
+ - docs/faq.md
263
+ - docs/getting-started.md
187
264
  - docs/getting-started/installing.md
188
265
  - docs/getting-started/tutorial.md
189
- - docs/summary.md
266
+ - docs/index.md
267
+ - docs/technical-overview.md
190
268
  - docs/technical-overview/concurrency.md
191
269
  - docs/technical-overview/design-principles.md
192
270
  - docs/technical-overview/exception-handling.md
193
271
  - docs/technical-overview/extending.md
194
- - docs/technical-overview/faq.md
195
272
  - docs/technical-overview/fiber-scheduling.md
273
+ - docs/user-guide.md
196
274
  - docs/user-guide/web-server.md
197
275
  - examples/core/01-spinning-up-fibers.rb
198
276
  - examples/core/02-awaiting-fibers.rb
199
277
  - examples/core/03-interrupting.rb
200
278
  - examples/core/xx-channels.rb
279
+ - examples/core/xx-deadlock.rb
201
280
  - examples/core/xx-deferring-an-operation.rb
202
281
  - examples/core/xx-erlang-style-genserver.rb
203
282
  - examples/core/xx-extended_fibers.rb
@@ -212,6 +291,7 @@ files:
212
291
  - examples/core/xx-signals.rb
213
292
  - examples/core/xx-sleeping.rb
214
293
  - examples/core/xx-spin_error_backtrace.rb
294
+ - examples/core/xx-state-machine.rb
215
295
  - examples/core/xx-supervisors.rb
216
296
  - examples/core/xx-thread-selector-sleep.rb
217
297
  - examples/core/xx-thread-selector-snooze.rb
@@ -220,8 +300,10 @@ files:
220
300
  - examples/core/xx-thread_pool.rb
221
301
  - examples/core/xx-throttling.rb
222
302
  - examples/core/xx-timeout.rb
303
+ - examples/core/xx-trace.rb
223
304
  - examples/core/xx-using-a-mutex.rb
224
305
  - examples/interfaces/pg_client.rb
306
+ - examples/interfaces/pg_notify.rb
225
307
  - examples/interfaces/pg_pool.rb
226
308
  - examples/interfaces/pg_transaction.rb
227
309
  - examples/interfaces/redis_channels.rb
@@ -239,6 +321,7 @@ files:
239
321
  - examples/io/xx-irb.rb
240
322
  - examples/io/xx-net-http.rb
241
323
  - examples/io/xx-open.rb
324
+ - examples/io/xx-switch.rb
242
325
  - examples/io/xx-system.rb
243
326
  - examples/io/xx-tcpserver.rb
244
327
  - examples/io/xx-tcpsocket.rb
@@ -268,6 +351,7 @@ files:
268
351
  - ext/gyro/socket.c
269
352
  - ext/gyro/thread.c
270
353
  - ext/gyro/timer.c
354
+ - ext/gyro/tracing.c
271
355
  - ext/libev/Changes
272
356
  - ext/libev/LICENSE
273
357
  - ext/libev/README
@@ -301,10 +385,12 @@ files:
301
385
  - lib/polyphony/extensions/socket.rb
302
386
  - lib/polyphony/extensions/thread.rb
303
387
  - lib/polyphony/fs.rb
388
+ - lib/polyphony/irb.rb
304
389
  - lib/polyphony/line_reader.rb
305
390
  - lib/polyphony/net.rb
306
391
  - lib/polyphony/postgres.rb
307
392
  - lib/polyphony/redis.rb
393
+ - lib/polyphony/trace.rb
308
394
  - lib/polyphony/version.rb
309
395
  - polyphony.gemspec
310
396
  - test/coverage.rb
@@ -323,14 +409,18 @@ files:
323
409
  - test/test_signal.rb
324
410
  - test/test_supervisor.rb
325
411
  - test/test_thread.rb
412
+ - test/test_thread_pool.rb
326
413
  - test/test_throttler.rb
327
414
  - test/test_timer.rb
415
+ - test/test_trace.rb
328
416
  homepage: https://dfab.gitbook.io/polyphony/
329
417
  licenses:
330
418
  - MIT
331
419
  metadata:
332
420
  source_code_uri: https://github.com/digital-fabric/polyphony
333
- documentation_uri: https://dfab.gitbook.io/polyphony/
421
+ documentation_uri: https://digital-fabric.github.io/polyphony/
422
+ homepage_uri: https://digital-fabric.github.io/polyphony/
423
+ changelog_uri: https://github.com/digital-fabric/polyphony/blob/master/CHANGELOG.md
334
424
  post_install_message:
335
425
  rdoc_options:
336
426
  - "--title"
@@ -350,7 +440,7 @@ required_rubygems_version: !ruby/object:Gem::Requirement
350
440
  - !ruby/object:Gem::Version
351
441
  version: '0'
352
442
  requirements: []
353
- rubygems_version: 3.0.6
443
+ rubygems_version: 3.1.2
354
444
  signing_key:
355
445
  specification_version: 4
356
446
  summary: 'Polyphony: Fiber-based Concurrency for Ruby'
data/docs/README.md DELETED
@@ -1,36 +0,0 @@
1
- # Polyphony - Easy Concurrency for Ruby
2
-
3
- > Polyphony \| pəˈlɪf\(ə\)ni \|
4
- > 1. _Music_ the style of simultaneously combining a number of parts, each forming an individual melody and harmonizing with each other.
5
- > 2. _Programming_ a Ruby gem for concurrent programming focusing on performance and developer happiness.
6
-
7
- Polyphony is a library for building concurrent applications in Ruby. Polyphony harnesses the power of [Ruby fibers](https://ruby-doc.org/core-2.5.1/Fiber.html) to provide a cooperative, sequential coroutine-based concurrency model. Under the hood, Polyphony uses [libev](https://github.com/enki/libev) as a high-performance event reactor that provides timers, I/O watchers and other asynchronous event primitives.
8
-
9
- Polyphony makes it possible to use normal Ruby built-in classes like `IO`, and `Socket` in a concurrent fashion without having to resort to threads. Polyphony takes care of context-switching automatically whenever a blocking call like `Socket#accept` or `IO#read` is issued.
10
-
11
- ## Features
12
-
13
- * Co-operative scheduling of concurrent tasks using Ruby fibers.
14
- * High-performance event reactor for handling I/O events and timers.
15
- * Natural, sequential programming style that makes it easy to reason about concurrent code.
16
- * Abstractions and constructs for controlling the execution of concurrent code: supervisors, cancel scopes, throttling, resource pools etc.
17
- * Code can use native networking classes and libraries, growing support for third-party gems such as `pg` and `redis`.
18
- * Use stdlib classes such as `TCPServer` and `TCPSocket` and `Net::HTTP`.
19
- * Competitive performance and scalability characteristics, in terms of both throughput and memory consumption.
20
-
21
- ## Prior Art
22
-
23
- Polyphony draws inspiration from the following, in no particular order:
24
-
25
- * [nio4r](https://github.com/socketry/nio4r/) and [async](https://github.com/socketry/async) (Polyphony's C-extension code is largely a spinoff of [nio4r's](https://github.com/socketry/nio4r/tree/master/ext))
26
- * [EventMachine](https://github.com/eventmachine/eventmachine)
27
- * [Trio](https://trio.readthedocs.io/)
28
- * [Erlang supervisors](http://erlang.org/doc/man/supervisor.html) (and actually, Erlang in general)
29
-
30
- ## Going further
31
-
32
- To learn more about using Polyphony to build concurrent applications, read the technical overview below, or look at the [included examples](https://github.com/digital-fabric/polyphony/tree/9e0f3b09213156bdf376ef33684ef267517f06e8/examples/README.md). A thorough reference is forthcoming.
33
-
34
- ## Contributing to Polyphony
35
-
36
- Issues and pull requests will be gladly accepted. Please use the git repository at https://github.com/digital-fabric/polyphony as your primary point of departure for contributing.
data/docs/summary.md DELETED
@@ -1,60 +0,0 @@
1
- # Table of contents
2
-
3
- * [Polyphony - Easy Concurrency for Ruby](../README.md)
4
-
5
- ## Getting Started
6
-
7
- * [Installing](getting-started/installing.md)
8
- * [Tutorial](getting-started/tutorial.md)
9
-
10
- ## Technical overview
11
-
12
- * [Design Principles](technical-overview/design-principles.md)
13
- * [Concurrency the Easy Way](technical-overview/concurrency.md)
14
- * [How Fibers are Scheduled](technical-overview/fiber-scheduling.md)
15
- * [Exception Handling](technical-overview/exception-handling.md)
16
- * [Frequently Asked Questions](technical-overview/faq.md)
17
-
18
- ## How To
19
-
20
- * [Make an echo server](howto/echo-server.md)
21
- * [Make an HTTP server](howto/http-server.md)
22
- * [Make a Websocket server](howto/websocket-server.md)
23
- * [Use timers](howto/timers.md)
24
- * [Throttle recurrent operations](howto/throttle.md)
25
- * [Cancel ongoing operations](howto/cancel.md)
26
- * [Control fibers](howto/fibers.md)
27
- * [Synchronize concurrent operations](howto/synchronize.md)
28
- * [Perform CPU-bound operations](howto/cpu-bound.md)
29
- * [Control backpressure](howto/backpressure.md)
30
- * [Fork worker processes](howto/worker-processes.md)
31
-
32
- ## Polyphony extensions
33
-
34
- * [Postgresql](extensions/pg)
35
- * [Redis](extensions/redis)
36
- * [IRB](extensions/irb)
37
- * [Throttlers](#)
38
- * [Resource Pools](#)
39
- * [Synchronisation](#)
40
- * [Web Server](user-guide/web-server.md)
41
- * [Websocket Server](#)
42
- * [Reactor API](#)
43
-
44
- ## API Reference
45
-
46
- * [Fiber](#)
47
- * [Gyro](#)
48
- * [Gyro::Async](#)
49
- * [Gyro::Child](#)
50
- * [Gyro::IO](#)
51
- * [Gyro::Timer](#)
52
- * [Kernel](#)
53
- * [Polyphony](#)
54
- * [Polyphony::CancelScope](#)
55
- * [Polyphony::Mutex](#)
56
- * [Polyphony::Pulser](#)
57
- * [Polyphony::ResourcePool](#)
58
- * [Polyphony::Throttler](#)
59
-
60
- ## [Contributing to Polyphony](contributing.md)
@@ -1,97 +0,0 @@
1
- # Frequently Asked Questions
2
-
3
- ## Why not just use callbacks instead of fibers?
4
-
5
- It is true that reactor engines such as libev use callbacks to handle events. There's also programming platforms such as [node.js](https://nodejs.org/) that base their entire API on the callback pattern. [EventMachine](https://www.rubydoc.info/gems/eventmachine/1.2.7) is a popular reactor library for Ruby that uses callbacks for handling events.
6
-
7
- Using callbacks means splitting your application logic into disjunct pieces of code. Consider the following example:
8
-
9
- ```ruby
10
- require 'eventmachine'
11
-
12
- module EchoServer
13
- def post_init
14
- puts '-- someone connected to the echo server!'
15
- end
16
-
17
- def receive_data data
18
- send_data ">>>you sent: #{data}"
19
- close_connection if data =~ /quit/i
20
- end
21
-
22
- def unbind
23
- puts '-- someone disconnected from the echo server!'
24
- end
25
- end
26
-
27
- # Note that this will block current thread.
28
- EventMachine.run {
29
- EventMachine.start_server '127.0.0.1', 8081, EchoServer
30
- }
31
- ```
32
-
33
- The client-handling code is split across three different callback methods. Compare this to the following equivalent using Polyphony:
34
-
35
- ```ruby
36
- require 'polyphony/auto_run'
37
-
38
- server = TCPServer.open('127.0.0.1', 8081)
39
- while (client = server.accept)
40
- spin do
41
- puts '-- someone connected to the echo server!'
42
- while (data = client.gets)
43
- client << ">>>you sent: #{data}"
44
- break if data =~ /quit/i
45
- end
46
- ensure
47
- client.close
48
- puts '-- someone disconnected from the echo server!'
49
- end
50
- end
51
- ```
52
-
53
- The Polyphony version is both more terse and explicit at the same time. It explicitly accepts connections on the server port, and the entire logic handling each client connection is contained in a single block. The order of the different actions - printing to the console, then echoing client messages, then finally closing the client connection and printing again to the console - is easy to grok. The echoing of client messages is also explicit: a simple loop waiting for a message, then responding to the client. In addition, we can use an `ensure` block to correctly cleanup even if exceptions are raised while handling the client.
54
-
55
- Using callbacks also makes it much more difficult to debug your program. when callbacks are used to handle events, the stack trace will necessarily start at the reactor, and thus lack any information about how the event came to be in the first place. Contrast this with Polyphony, where stack traces show the entire _sequence of events_ leading up to the present point in the code.
56
-
57
- In conclusion:
58
-
59
- * Callbacks cause the splitting of logic into disjunct chunks.
60
- * Callbacks do not provide a good error handling solution.
61
- * Callbacks often lead to code bloat.
62
- * Callbacks are harder to debug.
63
-
64
- ## If callbacks suck, why not use promises?
65
-
66
- Promises have gained a lot of traction during the last few years as an
67
- alternative to callbacks, above all in the Javascript community. While promises have been at a certain point considered for use in Polyphony, they were not found to offer enough of a benefit. Promises still cause split logic, are quite verbose and provide a non-native exception handling mechanism. In addition, they do not make it easier to debug your code.
68
-
69
- ## Why is awaiting implicit? Why not use explicit async/await?
70
-
71
- Actually, async/await was contemplated while developing Polyphony, but at a certain point it was decided to abandon these methods / decorators in favor of a more implicit approach. The most crucial issue with async/await is that it prevents the use of anything from Ruby's stdlib. Any operation involving stdlib classes needs to be wrapped in boilerplate.
72
-
73
- Instead, we have decided to make blocking operations implicit and thus allow the use of common APIs such as `Kernel#sleep` or `IO.popen` in a transparent manner. After all, these APIs in their stock form block execution just as well.
74
-
75
- ## Why use `Fiber#transfer` and not `Fiber#resume`?
76
-
77
- The API for `Fiber.yield`/`Fiber#resume` is stateful and is intended for the asymmetric execution of coroutines. This is useful when using generators, or other cases where one coroutine acts as a "server" and another as a "client". In Polyphony's case, all fibers are equal, and control can be transferred freely between them, which is much easier to achieve using `Fiber#transfer`. In addition, using `Fiber#transfer` allows us to perform blocking operations from the main fiber, which is not possible when using `Fiber#resume`.
78
-
79
- ## Why is Polyphony not split into multiple gems?
80
-
81
- Polyphony is currently at an experimental stage, and its different APIs are still in flux. For that reason, all the different parts of Polyphony are currently kept in a single gem. Once things stabilize, and as Polyphony approaches version 1.0, it will be split into separate gems, each with its own functionality.
82
-
83
- ## Can I use Polyphony in a multithreaded program?
84
-
85
- Not yet. We plan to support multiple threads when Ruby 3.0 is ready.
86
-
87
- ## Can I run Rails using Polyphony?
88
-
89
- Not yet. We do plan to support running Rails when our multithreaded support is ready (see above).
90
-
91
- ## How can I contribute to Polyphony?
92
-
93
- The Polyphony repository is at https://github.com/digital-fabric/polyphony. Feel free to create issues and contribute pull requests.
94
-
95
- ## Who is behind this project?
96
-
97
- I'm Sharon Rosner, an independent software developer living in France. Here's my [github profile](https://github.com/ciconia). You can contact me by writing to [noteflakes@gmail.com](mailto:ciconia@gmail.com).