concurrent_rails 0.1.1
Sign up to get free protection for your applications and to get access to all the features.
- checksums.yaml +7 -0
- data/MIT-LICENSE +20 -0
- data/README.md +138 -0
- data/Rakefile +15 -0
- data/lib/concurrent_rails.rb +9 -0
- data/lib/concurrent_rails/future.rb +45 -0
- data/lib/concurrent_rails/multi.rb +49 -0
- data/lib/concurrent_rails/railtie.rb +6 -0
- data/lib/concurrent_rails/version.rb +5 -0
- data/lib/tasks/concurrent_rails_tasks.rake +5 -0
- metadata +97 -0
checksums.yaml
ADDED
@@ -0,0 +1,7 @@
|
|
1
|
+
---
|
2
|
+
SHA256:
|
3
|
+
metadata.gz: ed13434b4d352006775b894399766908f80d8c259efa9ca8d690209d8a8b3ac6
|
4
|
+
data.tar.gz: 9ee145f8fa225b036e98ac2df6b2a96f6985859fcbbbe4a182faa5877ac55c9b
|
5
|
+
SHA512:
|
6
|
+
metadata.gz: cb4a0a4ff89170a8ff0318b4940d240a644301be81b68bb3319170c5b55450355432e167d37bb1e1fdefd96d163e7eae3aa40cec2a9197e44d856c04904309cd
|
7
|
+
data.tar.gz: f5bbedb78852ae590ccfbd5c87198af855b479c491cd82b827ede998ba99269a8d88e27bb97417b6f4d7df6ee5f14c23462c2fa0e5ad0e2affb5a5b8c64d11c1
|
data/MIT-LICENSE
ADDED
@@ -0,0 +1,20 @@
|
|
1
|
+
Copyright 2021 Luiz Eduardo
|
2
|
+
|
3
|
+
Permission is hereby granted, free of charge, to any person obtaining
|
4
|
+
a copy of this software and associated documentation files (the
|
5
|
+
"Software"), to deal in the Software without restriction, including
|
6
|
+
without limitation the rights to use, copy, modify, merge, publish,
|
7
|
+
distribute, sublicense, and/or sell copies of the Software, and to
|
8
|
+
permit persons to whom the Software is furnished to do so, subject to
|
9
|
+
the following conditions:
|
10
|
+
|
11
|
+
The above copyright notice and this permission notice shall be
|
12
|
+
included in all copies or substantial portions of the Software.
|
13
|
+
|
14
|
+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
|
15
|
+
EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
|
16
|
+
MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
|
17
|
+
NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE
|
18
|
+
LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
|
19
|
+
OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
|
20
|
+
WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
|
data/README.md
ADDED
@@ -0,0 +1,138 @@
|
|
1
|
+
# ConcurrentRails
|
2
|
+
|
3
|
+
![status](https://github.com/luizkowalski/concurrent_rails/actions/workflows/ruby.yml/badge.svg?branch=master)
|
4
|
+
|
5
|
+
Multithread is hard. [concurrent-ruby](https://github.com/ruby-concurrency/concurrent-ruby) did an amazing job
|
6
|
+
implementing the concepts of multithread in the Ruby world. The problem is that Rails doesn't play nice with it. Rails have a complex way of managing threads called Executor and concurrent-ruby (most specifically, [Future](https://github.com/ruby-concurrency/concurrent-ruby/blob/master/docs-source/future.md)) does not work seamlessly with it.
|
7
|
+
|
8
|
+
The goal of this gem is to provide a simple library that allows the developer to work with Futures without having to care about Rails's Executor and the whole pack of problems that come with it: autoload, thread pools, active record connections, etc.
|
9
|
+
|
10
|
+
## Usage
|
11
|
+
|
12
|
+
This library provides two classes that will help you run tasks in parallel: `ConcurrentRails::Future` and `ConcurrentRails::Multi`
|
13
|
+
|
14
|
+
### Future
|
15
|
+
|
16
|
+
`ConcurrentRails::Future` will execute your code in a separated thread and you can check the progress of it whenever you need. When the task is ready, you can access the result with `#result` function:
|
17
|
+
|
18
|
+
```ruby
|
19
|
+
irb(main):001:0> future = ConcurrentRails::Future.new do
|
20
|
+
sleep(5) # Simulate a long running task
|
21
|
+
42
|
22
|
+
end
|
23
|
+
|
24
|
+
# at this point, nothing has happened yet.
|
25
|
+
|
26
|
+
irb(main):002:0> future.execute
|
27
|
+
|
28
|
+
irb(main):003:0> future.state
|
29
|
+
=> :processing
|
30
|
+
|
31
|
+
# after 5 seconds
|
32
|
+
irb(main):004:0> future.state
|
33
|
+
=> :fulfilled
|
34
|
+
|
35
|
+
irb(main):005:0> future.value
|
36
|
+
=> 42
|
37
|
+
```
|
38
|
+
|
39
|
+
A task can also fail. In this case, the state of the future will be `rejected` and the exception can be accessed by invoking `reason`
|
40
|
+
|
41
|
+
```ruby
|
42
|
+
irb(main):001:1* future = ConcurrentRails::Future.new do
|
43
|
+
irb(main):002:1* 2 / 0
|
44
|
+
irb(main):003:0> end.execute
|
45
|
+
|
46
|
+
=> #<ConcurrentRails::Future...
|
47
|
+
|
48
|
+
irb(main):004:0> future.state
|
49
|
+
=> :rejected
|
50
|
+
|
51
|
+
irb(main):005:0> future.reason
|
52
|
+
=> #<ZeroDivisionError: divided by 0>
|
53
|
+
```
|
54
|
+
|
55
|
+
### Multi
|
56
|
+
|
57
|
+
`ConcurrentRails::Multi` will let you execute multiple tasks in parallel and aggregate the results of each task when they are done. `Multi` accepts an undefined number of `Proc`s.
|
58
|
+
|
59
|
+
```ruby
|
60
|
+
irb(main):001:1* multi = ConcurrentRails::Multi.enqueue(
|
61
|
+
irb(main):002:1* -> { 42 },
|
62
|
+
irb(main):003:1* -> { :multi_test }
|
63
|
+
irb(main):004:0> )
|
64
|
+
|
65
|
+
=> #<ConcurrentRails::Multi:0x00007fbc3f9ca3f8 @actions=[#<Proc:0x00007fbc3f9ca470..
|
66
|
+
irb(main):005:0> multi.complete?
|
67
|
+
=> true
|
68
|
+
|
69
|
+
irb(main):006:0> multi.compute
|
70
|
+
=> [42, :multi_test]
|
71
|
+
```
|
72
|
+
|
73
|
+
Given the fact that you can send any number of `Proc`s, the result from `compute` will always be an array, even if you provide only one proc.
|
74
|
+
|
75
|
+
```ruby
|
76
|
+
irb(main):007:1* multi = ConcurrentRails::Multi.enqueue(
|
77
|
+
irb(main):008:1* -> { 42 }
|
78
|
+
irb(main):009:0> )
|
79
|
+
=> #<ConcurrentRails::Multi:0x00007fbc403f0b98 @actions=[#<Proc:0x00007...
|
80
|
+
|
81
|
+
irb(main):010:0> multi.compute
|
82
|
+
=> [42]
|
83
|
+
```
|
84
|
+
|
85
|
+
Same as `Future`, one of the `Multi` tasks can fail. You can access the exception by calling `#errors`:
|
86
|
+
|
87
|
+
```ruby
|
88
|
+
irb(main):001:1* multi = ConcurrentRails::Multi.enqueue(
|
89
|
+
irb(main):002:1* -> { 42 },
|
90
|
+
irb(main):003:1* -> { 2 / 0 }
|
91
|
+
irb(main):004:0> )
|
92
|
+
=> #<ConcurrentRails::Multi:0x00007fb46d3ee3a0 @actions=[#<Proc:0x00007..
|
93
|
+
|
94
|
+
irb(main):005:0> multi.complete?
|
95
|
+
=> true
|
96
|
+
|
97
|
+
irb(main):006:0> multi.compute
|
98
|
+
=> [42, nil]
|
99
|
+
|
100
|
+
irb(main):007:0> multi.errors
|
101
|
+
=> [#<ZeroDivisionError: divided by 0>]
|
102
|
+
```
|
103
|
+
|
104
|
+
It is worth mention that a failed proc will return `nil`.
|
105
|
+
|
106
|
+
For more information on how Futures work and how Rails handle multithread check these links:
|
107
|
+
|
108
|
+
[Future documentation](https://github.com/ruby-concurrency/concurrent-ruby/blob/master/docs-source/future.md)
|
109
|
+
|
110
|
+
[Threading and code execution on rails](https://guides.rubyonrails.org/threading_and_code_execution.html)
|
111
|
+
|
112
|
+
## Installation
|
113
|
+
|
114
|
+
Add this line to your application's Gemfile:
|
115
|
+
|
116
|
+
```ruby
|
117
|
+
gem 'concurrent_rails'
|
118
|
+
```
|
119
|
+
|
120
|
+
And then execute:
|
121
|
+
|
122
|
+
```bash
|
123
|
+
$ bundle
|
124
|
+
```
|
125
|
+
|
126
|
+
Or install it yourself as:
|
127
|
+
|
128
|
+
```bash
|
129
|
+
$ gem install concurrent_rails
|
130
|
+
```
|
131
|
+
|
132
|
+
## Contributing
|
133
|
+
|
134
|
+
Pull-requests are always welcome
|
135
|
+
|
136
|
+
## License
|
137
|
+
|
138
|
+
The gem is available as open source under the terms of the [MIT License](https://opensource.org/licenses/MIT).
|
data/Rakefile
ADDED
@@ -0,0 +1,15 @@
|
|
1
|
+
# frozen_string_literal: true
|
2
|
+
|
3
|
+
require 'bundler/setup'
|
4
|
+
|
5
|
+
require 'bundler/gem_tasks'
|
6
|
+
|
7
|
+
require 'rake/testtask'
|
8
|
+
|
9
|
+
Rake::TestTask.new(:test) do |t|
|
10
|
+
t.libs << 'test'
|
11
|
+
t.pattern = 'test/**/*_test.rb'
|
12
|
+
t.verbose = false
|
13
|
+
end
|
14
|
+
|
15
|
+
task default: :test
|
@@ -0,0 +1,45 @@
|
|
1
|
+
# frozen_string_literal: true
|
2
|
+
|
3
|
+
module ConcurrentRails
|
4
|
+
class Future
|
5
|
+
extend Forwardable
|
6
|
+
|
7
|
+
def initialize(executor: :io, &block)
|
8
|
+
@task = block
|
9
|
+
@executor = executor
|
10
|
+
@future = run_on_rails(block)
|
11
|
+
end
|
12
|
+
|
13
|
+
def execute
|
14
|
+
future.execute
|
15
|
+
|
16
|
+
self
|
17
|
+
end
|
18
|
+
|
19
|
+
def value
|
20
|
+
Rails.application.executor.wrap do
|
21
|
+
result = nil
|
22
|
+
|
23
|
+
ActiveSupport::Dependencies.interlock.permit_concurrent_loads do
|
24
|
+
result = future.value
|
25
|
+
end
|
26
|
+
|
27
|
+
result
|
28
|
+
end
|
29
|
+
end
|
30
|
+
|
31
|
+
def_delegators :@future, :state, :reason, :rejected?, :complete?, :add_observer
|
32
|
+
|
33
|
+
private
|
34
|
+
|
35
|
+
def run_on_rails(block)
|
36
|
+
@future = Rails.application.executor.wrap do
|
37
|
+
Concurrent::Future.new(executor: executor) do
|
38
|
+
Rails.application.executor.wrap(&block)
|
39
|
+
end
|
40
|
+
end
|
41
|
+
end
|
42
|
+
|
43
|
+
attr_reader :executor, :task, :future
|
44
|
+
end
|
45
|
+
end
|
@@ -0,0 +1,49 @@
|
|
1
|
+
# frozen_string_literal: true
|
2
|
+
|
3
|
+
module ConcurrentRails
|
4
|
+
class Multi
|
5
|
+
def self.enqueue(*actions, executor: :io)
|
6
|
+
unless actions.all? { |action| action.is_a?(Proc) }
|
7
|
+
raise ArgumentError, '#enqueue accepts `Proc`s only'
|
8
|
+
end
|
9
|
+
|
10
|
+
new(actions, executor).enqueue
|
11
|
+
end
|
12
|
+
|
13
|
+
def initialize(actions, executor)
|
14
|
+
@actions = actions
|
15
|
+
@executor = executor
|
16
|
+
@exceptions = Concurrent::Array.new
|
17
|
+
end
|
18
|
+
|
19
|
+
def enqueue
|
20
|
+
@futures = actions.map do |action|
|
21
|
+
f = ConcurrentRails::Future.new(executor: executor, &action)
|
22
|
+
f.add_observer(self)
|
23
|
+
f.execute
|
24
|
+
end
|
25
|
+
|
26
|
+
self
|
27
|
+
end
|
28
|
+
|
29
|
+
def compute
|
30
|
+
futures.map(&:value)
|
31
|
+
end
|
32
|
+
|
33
|
+
def complete?
|
34
|
+
futures.all?(&:complete?)
|
35
|
+
end
|
36
|
+
|
37
|
+
def errors
|
38
|
+
@exceptions
|
39
|
+
end
|
40
|
+
|
41
|
+
private
|
42
|
+
|
43
|
+
def update(_time, _value, reason)
|
44
|
+
@exceptions << reason if reason
|
45
|
+
end
|
46
|
+
|
47
|
+
attr_reader :actions, :futures, :exceptions, :executor
|
48
|
+
end
|
49
|
+
end
|
metadata
ADDED
@@ -0,0 +1,97 @@
|
|
1
|
+
--- !ruby/object:Gem::Specification
|
2
|
+
name: concurrent_rails
|
3
|
+
version: !ruby/object:Gem::Version
|
4
|
+
version: 0.1.1
|
5
|
+
platform: ruby
|
6
|
+
authors:
|
7
|
+
- Luiz Eduardo
|
8
|
+
autorequire:
|
9
|
+
bindir: bin
|
10
|
+
cert_chain: []
|
11
|
+
date: 2021-04-18 00:00:00.000000000 Z
|
12
|
+
dependencies:
|
13
|
+
- !ruby/object:Gem::Dependency
|
14
|
+
name: rails
|
15
|
+
requirement: !ruby/object:Gem::Requirement
|
16
|
+
requirements:
|
17
|
+
- - ">="
|
18
|
+
- !ruby/object:Gem::Version
|
19
|
+
version: '5.0'
|
20
|
+
type: :runtime
|
21
|
+
prerelease: false
|
22
|
+
version_requirements: !ruby/object:Gem::Requirement
|
23
|
+
requirements:
|
24
|
+
- - ">="
|
25
|
+
- !ruby/object:Gem::Version
|
26
|
+
version: '5.0'
|
27
|
+
- !ruby/object:Gem::Dependency
|
28
|
+
name: rubocop
|
29
|
+
requirement: !ruby/object:Gem::Requirement
|
30
|
+
requirements:
|
31
|
+
- - '='
|
32
|
+
- !ruby/object:Gem::Version
|
33
|
+
version: '1.12'
|
34
|
+
type: :runtime
|
35
|
+
prerelease: false
|
36
|
+
version_requirements: !ruby/object:Gem::Requirement
|
37
|
+
requirements:
|
38
|
+
- - '='
|
39
|
+
- !ruby/object:Gem::Version
|
40
|
+
version: '1.12'
|
41
|
+
- !ruby/object:Gem::Dependency
|
42
|
+
name: rubocop-performance
|
43
|
+
requirement: !ruby/object:Gem::Requirement
|
44
|
+
requirements:
|
45
|
+
- - '='
|
46
|
+
- !ruby/object:Gem::Version
|
47
|
+
version: '1.10'
|
48
|
+
type: :runtime
|
49
|
+
prerelease: false
|
50
|
+
version_requirements: !ruby/object:Gem::Requirement
|
51
|
+
requirements:
|
52
|
+
- - '='
|
53
|
+
- !ruby/object:Gem::Version
|
54
|
+
version: '1.10'
|
55
|
+
description: Small library to make concurrent-ruby and Rails play nice together
|
56
|
+
email:
|
57
|
+
- luizeduardokowalski@gmail.com
|
58
|
+
executables: []
|
59
|
+
extensions: []
|
60
|
+
extra_rdoc_files: []
|
61
|
+
files:
|
62
|
+
- MIT-LICENSE
|
63
|
+
- README.md
|
64
|
+
- Rakefile
|
65
|
+
- lib/concurrent_rails.rb
|
66
|
+
- lib/concurrent_rails/future.rb
|
67
|
+
- lib/concurrent_rails/multi.rb
|
68
|
+
- lib/concurrent_rails/railtie.rb
|
69
|
+
- lib/concurrent_rails/version.rb
|
70
|
+
- lib/tasks/concurrent_rails_tasks.rake
|
71
|
+
homepage: https://github.com/luizkowalski/concurrent-rails
|
72
|
+
licenses:
|
73
|
+
- MIT
|
74
|
+
metadata:
|
75
|
+
homepage_uri: https://github.com/luizkowalski/concurrent-rails
|
76
|
+
source_code_uri: https://github.com/luizkowalski/concurrent-rails
|
77
|
+
changelog_uri: https://github.com/luizkowalski/concurrent-rails/CHANGELOG
|
78
|
+
post_install_message:
|
79
|
+
rdoc_options: []
|
80
|
+
require_paths:
|
81
|
+
- lib
|
82
|
+
required_ruby_version: !ruby/object:Gem::Requirement
|
83
|
+
requirements:
|
84
|
+
- - ">="
|
85
|
+
- !ruby/object:Gem::Version
|
86
|
+
version: '2.4'
|
87
|
+
required_rubygems_version: !ruby/object:Gem::Requirement
|
88
|
+
requirements:
|
89
|
+
- - ">="
|
90
|
+
- !ruby/object:Gem::Version
|
91
|
+
version: '0'
|
92
|
+
requirements: []
|
93
|
+
rubygems_version: 3.1.6
|
94
|
+
signing_key:
|
95
|
+
specification_version: 4
|
96
|
+
summary: Multithread is hard
|
97
|
+
test_files: []
|