polyphony 0.99 → 0.99.1

Sign up to get free protection for your applications and to get access to all the features.
Files changed (94) hide show
  1. checksums.yaml +4 -4
  2. data/.github/FUNDING.yml +1 -1
  3. data/.rubocop.yml +3 -3
  4. data/.yardopts +30 -0
  5. data/CHANGELOG.md +4 -0
  6. data/LICENSE +1 -1
  7. data/README.md +63 -29
  8. data/Rakefile +1 -5
  9. data/TODO.md +0 -4
  10. data/docs/{main-concepts/concurrency.md → concurrency.md} +2 -9
  11. data/docs/{main-concepts/design-principles.md → design-principles.md} +3 -9
  12. data/docs/{main-concepts/exception-handling.md → exception-handling.md} +2 -9
  13. data/docs/{main-concepts/extending.md → extending.md} +2 -9
  14. data/docs/faq.md +3 -16
  15. data/docs/{main-concepts/fiber-scheduling.md → fiber-scheduling.md} +1 -9
  16. data/docs/link_rewriter.rb +16 -0
  17. data/docs/{getting-started/overview.md → overview.md} +1 -30
  18. data/docs/{getting-started/tutorial.md → tutorial.md} +3 -28
  19. data/docs/{_posts/2020-07-26-polyphony-0.44.md → whats-new.md} +3 -1
  20. data/examples/adapters/redis_client.rb +3 -2
  21. data/examples/io/echo_server.rb +1 -1
  22. data/examples/io/echo_server_plain_ruby.rb +26 -0
  23. data/ext/polyphony/backend_io_uring.c +154 -9
  24. data/ext/polyphony/backend_io_uring_context.c +21 -12
  25. data/ext/polyphony/backend_io_uring_context.h +12 -7
  26. data/ext/polyphony/backend_libev.c +1 -1
  27. data/ext/polyphony/extconf.rb +24 -8
  28. data/ext/polyphony/fiber.c +79 -2
  29. data/ext/polyphony/io_extensions.c +53 -0
  30. data/ext/polyphony/pipe.c +42 -2
  31. data/ext/polyphony/polyphony.c +345 -31
  32. data/ext/polyphony/polyphony.h +9 -2
  33. data/ext/polyphony/queue.c +181 -0
  34. data/ext/polyphony/ring_buffer.c +0 -1
  35. data/ext/polyphony/runqueue.c +8 -1
  36. data/ext/polyphony/runqueue_ring_buffer.c +13 -0
  37. data/ext/polyphony/runqueue_ring_buffer.h +2 -1
  38. data/ext/polyphony/socket_extensions.c +6 -0
  39. data/ext/polyphony/thread.c +34 -2
  40. data/lib/polyphony/adapters/process.rb +11 -1
  41. data/lib/polyphony/adapters/sequel.rb +1 -1
  42. data/lib/polyphony/core/channel.rb +2 -0
  43. data/lib/polyphony/core/debug.rb +1 -1
  44. data/lib/polyphony/core/global_api.rb +25 -24
  45. data/lib/polyphony/core/resource_pool.rb +7 -6
  46. data/lib/polyphony/core/sync.rb +2 -2
  47. data/lib/polyphony/core/thread_pool.rb +3 -3
  48. data/lib/polyphony/core/timer.rb +8 -8
  49. data/lib/polyphony/extensions/exception.rb +2 -0
  50. data/lib/polyphony/extensions/fiber.rb +15 -13
  51. data/lib/polyphony/extensions/io.rb +127 -5
  52. data/lib/polyphony/extensions/kernel.rb +20 -2
  53. data/lib/polyphony/extensions/openssl.rb +100 -11
  54. data/lib/polyphony/extensions/pipe.rb +103 -7
  55. data/lib/polyphony/extensions/process.rb +13 -1
  56. data/lib/polyphony/extensions/socket.rb +93 -27
  57. data/lib/polyphony/extensions/thread.rb +9 -1
  58. data/lib/polyphony/extensions/timeout.rb +1 -1
  59. data/lib/polyphony/version.rb +2 -1
  60. data/lib/polyphony.rb +27 -7
  61. data/polyphony.gemspec +1 -8
  62. data/test/stress.rb +1 -1
  63. data/test/test_global_api.rb +45 -7
  64. data/test/test_socket.rb +96 -0
  65. data/test/test_timer.rb +5 -5
  66. metadata +17 -40
  67. data/docs/_config.yml +0 -64
  68. data/docs/_includes/head.html +0 -40
  69. data/docs/_includes/title.html +0 -1
  70. data/docs/_sass/custom/custom.scss +0 -10
  71. data/docs/_sass/overrides.scss +0 -0
  72. data/docs/api-reference/exception.md +0 -31
  73. data/docs/api-reference/fiber.md +0 -425
  74. data/docs/api-reference/index.md +0 -9
  75. data/docs/api-reference/io.md +0 -36
  76. data/docs/api-reference/object.md +0 -99
  77. data/docs/api-reference/polyphony-baseexception.md +0 -33
  78. data/docs/api-reference/polyphony-cancel.md +0 -26
  79. data/docs/api-reference/polyphony-moveon.md +0 -24
  80. data/docs/api-reference/polyphony-net.md +0 -20
  81. data/docs/api-reference/polyphony-process.md +0 -28
  82. data/docs/api-reference/polyphony-resourcepool.md +0 -59
  83. data/docs/api-reference/polyphony-restart.md +0 -18
  84. data/docs/api-reference/polyphony-terminate.md +0 -18
  85. data/docs/api-reference/polyphony-threadpool.md +0 -67
  86. data/docs/api-reference/polyphony-throttler.md +0 -77
  87. data/docs/api-reference/polyphony.md +0 -36
  88. data/docs/api-reference/thread.md +0 -88
  89. data/docs/favicon.ico +0 -0
  90. data/docs/getting-started/index.md +0 -10
  91. data/docs/getting-started/installing.md +0 -34
  92. /data/{docs/assets/img → assets}/echo-fibers.svg +0 -0
  93. /data/{docs → assets}/polyphony-logo.png +0 -0
  94. /data/{docs/assets/img → assets}/sleeping-fiber.svg +0 -0
@@ -1,59 +0,0 @@
1
- ---
2
- layout: page
3
- title: Polyphony::ResourcePool
4
- parent: API Reference
5
- permalink: /api-reference/polyphony-resourcepool/
6
- ---
7
- # Polyphony::ResourcePool
8
-
9
- `Polyphony::ResourcePool` implements a general purpose resource pool for
10
- limiting concurrent access to a resource or multiple copies thereof. A resource
11
- pool might be used for example to limit the number of concurrent database
12
- connections.
13
-
14
- ## Class methods
15
-
16
- ## Instance methods
17
-
18
- ### #acquire({ block })
19
-
20
- Acquires a resource and passes it to the given block. The resource will be used
21
- exclusively by the given block, and then returned to the pool. This method
22
- blocks until the given block has completed running. If no resource is available,
23
- this method blocks until a resource has been released.
24
-
25
- ```ruby
26
- db_connections = Polyphony::ResourcePool.new(limit: 5) { PG.connect(opts) }
27
-
28
- def query_records(sql)
29
- db_connections.acquire do |db|
30
- db.query(sql).to_a
31
- end
32
- end
33
- ```
34
-
35
- ### #available → count
36
-
37
- Returns the number of resources currently available in the resource pool.
38
-
39
- ### #initialize(limit: number, { block })
40
-
41
- Initializes a new resource pool with the given maximum number of concurrent
42
- resources. The given block is used to create the resource.
43
-
44
- ```ruby
45
- require 'postgres'
46
-
47
- opts = { host: '/tmp', user: 'admin', dbname: 'mydb' }
48
- db_connections = Polyphony::ResourcePool.new(limit: 5) { PG.connect(opts) }
49
- ```
50
-
51
- ### #limit → count
52
-
53
- Returns the size limit of the resource pool.
54
-
55
- ### #size → count
56
-
57
- Returns the total number of allocated resources in the resource pool. This
58
- includes both available and unavailable resources.
59
-
@@ -1,18 +0,0 @@
1
- ---
2
- layout: page
3
- title: Polyphony::Restart
4
- parent: API Reference
5
- permalink: /api-reference/polyphony-restart/
6
- ---
7
- # Polyphony::Restart
8
-
9
- `Polyphony::Restart` is an exception class used to restart a fiber. Applications
10
- will not normally raise a `Polyphony::Restart` exception, but would rather use
11
- `Fiber#restart`.
12
-
13
- ```ruby
14
- f = spin { do_something_slow }
15
- ...
16
- f.restart
17
- ...
18
- ```
@@ -1,18 +0,0 @@
1
- ---
2
- layout: page
3
- title: Polyphony::Terminate
4
- parent: API Reference
5
- permalink: /api-reference/polyphony-terminate/
6
- ---
7
- # Polyphony::Terminate
8
-
9
- `Polyphony::Terminate` is an exception class used to terminate a fiber without
10
- propagating the exception. It should never be rescued. A `Polyphony::Terminate`
11
- exception is normally raised using APIs such as `Fiber#terminate` or
12
- `Fiber#terminate_all_children`.
13
-
14
- ```ruby
15
- f = spin { do_something_slow }
16
- ...
17
- f.terminate
18
- ```
@@ -1,67 +0,0 @@
1
- ---
2
- layout: page
3
- title: Polyphony::ThreadPool
4
- parent: API Reference
5
- permalink: /api-reference/polyphony-threadpool/
6
- ---
7
- # Polyphony::ThreadPool
8
-
9
- `Polyphony::ThreadPool` implements a general purpose thread pool, normally used
10
- for the execution of non-fiber aware operations, such as C-extension based
11
- third-party libraries or other system call blocking APIs. The Polyphony
12
- implementation of a thread pool allows limiting the number of threads used for
13
- performing a recurring operation across one or more fibers.
14
-
15
- A default thread pool is available for quick access to this feature.
16
-
17
- ## Class methods
18
-
19
- ### #process({ block }) → object
20
-
21
- Runs the given block on the default thread pool. The default pool will be
22
- created on the first call to `#process`. This method will block until the
23
- operation has completed. The return value is that of the given block. Any
24
- uncaught exception will be propagated to the callsite.
25
-
26
- ```ruby
27
- result = Polyphony::ThreadPool.process { lengthy_op }
28
- ```
29
-
30
- ## Instance methods
31
-
32
- ### #busy? → true or false
33
-
34
- Returns true if operations are currently being run on the thread pool.
35
-
36
- ### cast({ block }) → pool
37
-
38
- Runs the given block on one of the threads in the pool in a fire-and-forget
39
- manner, without waiting for the operation to complete. Using `#cast` to run an
40
- operation means there's no way of knowing if the operation has completed or if
41
- any exception has been raised, other than inside the block.
42
-
43
- ```ruby
44
- my_pool.cast { puts 'Hello world' }
45
- do_something_else
46
- ```
47
-
48
- ### #initialize(size = Etc.nprocessors)
49
-
50
- Initializes a new instance of `Polyphony::ThreadPool` with the given maximum
51
- number of threads. The default size is the number of available processors
52
- (number of CPU cores).
53
-
54
- ```ruby
55
- my_pool = Polyphony::ThreadPool.new(3)
56
- ```
57
-
58
- ### #process({ block }) → object
59
-
60
- Runs the given block on one of the threads in the thread pool and blocks until
61
- the operation has completed. The return value is that of the given block. Any
62
- uncaught exception will be propagated to the callsite.
63
-
64
- ```ruby
65
- pool = Polyphony::ThreadPool.new(3)
66
- result = pool.process { lengthy_op }
67
- ```
@@ -1,77 +0,0 @@
1
- ---
2
- layout: page
3
- title: Polyphony::Throttler
4
- parent: API Reference
5
- permalink: /api-reference/polyphony-throttler/
6
- ---
7
- # Polyphony::Throttler
8
-
9
- `Polyphony::Throttler` implements general purpose operation throttling, or rate
10
- limiting. A `Polyphony::Throttler` instance may be used to limit the rate of an
11
- arbitrary operation in a single fiber, or across multiple fibers. For example,
12
- an HTTP server can limit the number of requests per second across for each
13
- client, or for all clients.
14
-
15
- A throttler is invoked using its `#call` method, e.g.:
16
-
17
- ```ruby
18
- # throttle rate: one per second
19
- throttler = Polyphony::Throttler.new(1)
20
-
21
- 10.times do |i|
22
- spin_loop { throttler.call { p [i, Time.now] } }
23
- end
24
- ```
25
-
26
- If many throttler instances are created over the application's lifetime, they
27
- should be stopped using the `#stop` method in order to prevent memory leaks.
28
- This is best done using an `ensure` block:
29
-
30
- ```ruby
31
- def start_server
32
- throttler = Polyphony::Throttler.new(1000)
33
- MyServer.start do |req|
34
- throttler.call { handle_request(req) }
35
- end
36
- ensure
37
- throttler.stop
38
- end
39
- ```
40
-
41
- ## Instance methods
42
-
43
- ### #initialize(rate)<br>#initialize(interval: interval)<br>#initialize(rate: rate)
44
-
45
- Initializes the throttler with the given rate. The rate can be specified either
46
- as a number signifying the maximum rate per second, or as a keyword argument. If
47
- the rate is specified using the `interval:` keyword argument, the value given is
48
- the minimum interval between consecutive invocations.
49
-
50
- ```ruby
51
- # These are all equivalent
52
- Polyphony::Throttler.new(10)
53
- Polyphony::Throttler.new(rate: 10)
54
- Polyphony::Throttler.new(interval: 0.1)
55
- ```
56
-
57
- ### #call({ block }) → object
58
-
59
- Invokes the throttler with the given block. This method will sleep for an
60
- interval of time required to throttle the execution of the given block. The
61
- return value is the return value of the given block.
62
-
63
- ### #stop → throttler
64
-
65
- Stops the timer associated with the throttler. This method should be called when
66
- the throttler is no longer needed. This is best done from an `ensure` block.
67
-
68
- ```ruby
69
- def start_server
70
- throttler = Polyphony::Throttler.new(1000)
71
- MyServer.start do |req|
72
- throttler.call { handle_request(req) }
73
- end
74
- ensure
75
- throttler.stop
76
- end
77
- ```
@@ -1,36 +0,0 @@
1
- ---
2
- layout: page
3
- title: Polyphony
4
- parent: API Reference
5
- permalink: /api-reference/polyphony/
6
- ---
7
- # Polyphony
8
-
9
- The `Polyphony` module acts as a namespace containing general Polyphony
10
- functionalities.
11
-
12
- ## Class Methods
13
-
14
- ### #emit_signal_exception(exception, fiber = Thread.main.main_fiber) → thread
15
-
16
- Emits an exception to the given fiber from a signal handler.
17
-
18
- ### #fork({ block }) → pid
19
-
20
- Forks a child process running the given block. Due to the way Ruby implements
21
- fibers, along with how signals interact with them, Polyphony-based applications
22
- should use `Polyphony#fork` rather than `Kernel#fork`. In order to continue
23
- handling fiber scheduling and signal handling correctly, the child process does
24
- the following:
25
-
26
- - A new fiber is created using `Fiber#new` and control is transferred to it.
27
- - Notify the event loop that a fork has occurred (by calling `ev_loop_fork`).
28
- - Setup the current fiber as the main thread's main fiber.
29
- - Setup fiber scheduling for the main thread.
30
- - Install fiber-aware signal handlers for the `TERM` and `INT` signals.
31
- - Run the block.
32
- - Correctly handle uncaught exceptions, including `SystemExit` and `Interrupt`.
33
-
34
- ### #watch_process(cmd = nil, { block })
35
-
36
- Alternative for [`Polyphony::Process.watch`](../polyphony-process/#watchcmd--nil--block-).
@@ -1,88 +0,0 @@
1
- ---
2
- layout: page
3
- title: ::Thread
4
- parent: API Reference
5
- permalink: /api-reference/thread/
6
- ---
7
- # ::Thread
8
-
9
- [Ruby core Thread documentation](https://ruby-doc.org/core-2.7.0/Thread.html)
10
-
11
- Polyphony enhances the core `Thread` class with APIs for switching and
12
- scheduling fibers, and reimplements some of its APIs such as `Thread#raise`
13
- using fibers which, incidentally, make it safe.
14
-
15
- Each thread has its own run queue and its own system backend. While running
16
- multiple threads does not result in true parallelism in MRI Ruby, sometimes
17
- multithreading is inevitable, for instance when using third-party gems that
18
- spawn threads, or when calling blocking APIs that are not fiber-aware.
19
-
20
- ## Class Methods
21
-
22
- ## Instance methods
23
-
24
- ### #&lt;&lt;(object) → fiber<br>#send(object) → fiber
25
-
26
- Sends a message to the thread's main fiber. For further details see
27
- [`Fiber#<<`](../fiber/#object--fibersendobject--fiber).
28
-
29
- ### #fiber_scheduling_stats → stats
30
-
31
- Returns statistics relating to fiber scheduling for the thread with the
32
- following entries:
33
-
34
- - `:scheduled_fibers` - number of fibers currently in the run queue
35
- - `:pending_watchers` - number of currently pending event watchers
36
-
37
- ### #join → object<br>#await → object
38
-
39
- Waits for the thread to finish running. If the thread has terminated with an
40
- uncaught exception, it will be reraised in the context of the calling fiber. If
41
- no excecption is raised, returns the thread's result.
42
-
43
- ```ruby
44
- t = Thread.new { sleep 1 }
45
- t.join
46
- ```
47
-
48
- ### #main_fiber → fiber
49
-
50
- Returns the main fiber for the thread.
51
-
52
- ### #result → object
53
-
54
- Returns the result of the thread's main fiber.
55
-
56
- ```ruby
57
- t = Thread.new { 'foo' }
58
- t.join
59
- t.result #=> 'foo'
60
- ```
61
-
62
- ### #switch_fiber
63
-
64
- invokes a switchpoint, selecting and resuming the next fiber to run. The
65
- switching algorithm works as follows:
66
-
67
- - If the run queue is not empty, conditionally run the event loop a single time
68
- in order to prevent event starvation when there's always runnable fibers
69
- waiting to be resumed.
70
- - If the run queue is empty, run the event loop until a fiber is put on the run
71
- queue.
72
- - Switch to the first fiber in the run queue.
73
-
74
- This method is normally not called directly by the application. Calling
75
- `Thread#switch_fiber` means the current fiber has no more work to do and would
76
- like yield to other fibers. Note that if the current fiber needs to resume at a
77
- later time, it should be scheduled before calling `Thread#switch_fiber`.
78
-
79
- ```ruby
80
- # schedule current fiber to be resumed later
81
- Fiber.current.schedule
82
-
83
- # switch to another fiber
84
- Thread.current.switch_fiber
85
-
86
- # the fiber is resumed
87
- resume_work
88
- ```
data/docs/favicon.ico DELETED
Binary file
@@ -1,10 +0,0 @@
1
- ---
2
- layout: page
3
- title: Getting Started
4
- description: Getting started with Polyphony
5
- has_children: true
6
- nav_order: 2
7
- ---
8
-
9
- # Getting Started
10
- {: .no_toc }
@@ -1,34 +0,0 @@
1
- ---
2
- layout: page
3
- title: Installing Polyphony
4
- parent: Getting Started
5
- nav_order: 1
6
- ---
7
- # Installing Polyphony
8
-
9
- ## System Requirements
10
-
11
- In order to use Polyphony you need to have:
12
-
13
- - Linux or MacOS (support for Windows will come at a later stage)
14
- - Ruby (MRI) 2.6 or newer
15
-
16
- ## Installing the Polyphony Gem
17
-
18
- Add this line to your application's Gemfile:
19
-
20
- ```ruby
21
- gem 'polyphony'
22
- ```
23
-
24
- And then execute:
25
-
26
- ```bash
27
- $ bundle
28
- ```
29
-
30
- Or install it yourself as:
31
-
32
- ```bash
33
- $ gem install polyphony
34
- ```
File without changes
File without changes
File without changes