good_job 0.4.0 → 0.8.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/CHANGELOG.md +44 -3
- data/README.md +96 -12
- data/lib/active_job/queue_adapters/good_job_adapter.rb +19 -0
- data/lib/good_job.rb +2 -0
- data/lib/good_job/adapter.rb +30 -9
- data/lib/good_job/cli.rb +45 -9
- data/lib/good_job/job.rb +30 -4
- data/lib/good_job/lockable.rb +7 -2
- data/lib/good_job/scheduler.rb +10 -22
- data/lib/good_job/version.rb +1 -1
- metadata +31 -2
checksums.yaml
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
---
|
2
2
|
SHA256:
|
3
|
-
metadata.gz:
|
4
|
-
data.tar.gz:
|
3
|
+
metadata.gz: 9d3f717a6ef87a75232173cd29e90842edc441808036116ced1355d0c51957a2
|
4
|
+
data.tar.gz: df7aaaab677397d52694697df2016bc9fa7c65db6bebe4f3a112432b5b9f3a96
|
5
5
|
SHA512:
|
6
|
-
metadata.gz:
|
7
|
-
data.tar.gz:
|
6
|
+
metadata.gz: ec049cfa4441c5bfa61b70661ed6543cf22e9e3625036dd5ce824f528a2516347961dd95b1ecccc444587f5a31a353499f50a362005c43ea30263b9bb51d1524
|
7
|
+
data.tar.gz: 4e0c0adbee22e2e8ae825015917cb52fe15475d409d48eec0a43df659de5fca95447f611bcacddcb98095888f8a809136890bed0d1fa9465b7bed38b5b38295b
|
data/CHANGELOG.md
CHANGED
@@ -1,8 +1,49 @@
|
|
1
1
|
# Changelog
|
2
2
|
|
3
|
-
## [
|
3
|
+
## [v0.8.1](https://github.com/bensheldon/good_job/tree/v0.8.1) (2020-07-17)
|
4
4
|
|
5
|
-
[Full Changelog](https://github.com/bensheldon/good_job/compare/v0.
|
5
|
+
[Full Changelog](https://github.com/bensheldon/good_job/compare/v0.6.0...v0.8.1)
|
6
|
+
|
7
|
+
**Closed issues:**
|
8
|
+
|
9
|
+
- Always store a default priority \(0\) and scheduled\_at\(Time.current\) [\#30](https://github.com/bensheldon/good_job/issues/30)
|
10
|
+
|
11
|
+
**Merged pull requests:**
|
12
|
+
|
13
|
+
- Move where\(scheduled\_at: Time.current\) into dynamic part of GoodJob::Job::Performer [\#42](https://github.com/bensheldon/good_job/pull/42) ([bensheldon](https://github.com/bensheldon))
|
14
|
+
- Replace Adapter inline boolean kwarg with execution\_mode instead [\#41](https://github.com/bensheldon/good_job/pull/41) ([bensheldon](https://github.com/bensheldon))
|
15
|
+
- Add more examples to Readme [\#39](https://github.com/bensheldon/good_job/pull/39) ([bensheldon](https://github.com/bensheldon))
|
16
|
+
- Add additional Rubocops and lint [\#38](https://github.com/bensheldon/good_job/pull/38) ([bensheldon](https://github.com/bensheldon))
|
17
|
+
- Always store a default queue\_name, priority and scheduled\_at; index by queue\_name and scheduled\_at [\#37](https://github.com/bensheldon/good_job/pull/37) ([bensheldon](https://github.com/bensheldon))
|
18
|
+
|
19
|
+
## [v0.6.0](https://github.com/bensheldon/good_job/tree/v0.6.0) (2020-07-15)
|
20
|
+
|
21
|
+
[Full Changelog](https://github.com/bensheldon/good_job/compare/v0.5.0...v0.6.0)
|
22
|
+
|
23
|
+
**Closed issues:**
|
24
|
+
|
25
|
+
- Improve the command line options [\#32](https://github.com/bensheldon/good_job/issues/32)
|
26
|
+
- Allow config.active\_job.queue\_adapter = :good\_job to work [\#5](https://github.com/bensheldon/good_job/issues/5)
|
27
|
+
|
28
|
+
**Merged pull requests:**
|
29
|
+
|
30
|
+
- Improve generation of changelog [\#36](https://github.com/bensheldon/good_job/pull/36) ([bensheldon](https://github.com/bensheldon))
|
31
|
+
- Update Github Action Workflow for Backlog Project Board [\#35](https://github.com/bensheldon/good_job/pull/35) ([bensheldon](https://github.com/bensheldon))
|
32
|
+
- Add configuration options to good\_job executable [\#33](https://github.com/bensheldon/good_job/pull/33) ([bensheldon](https://github.com/bensheldon))
|
33
|
+
- Extract Job querying behavior out of Scheduler [\#31](https://github.com/bensheldon/good_job/pull/31) ([bensheldon](https://github.com/bensheldon))
|
34
|
+
- Allow configuration of Rails queue adapter with `:good\_job` [\#28](https://github.com/bensheldon/good_job/pull/28) ([bensheldon](https://github.com/bensheldon))
|
35
|
+
|
36
|
+
## [v0.5.0](https://github.com/bensheldon/good_job/tree/v0.5.0) (2020-07-13)
|
37
|
+
|
38
|
+
[Full Changelog](https://github.com/bensheldon/good_job/compare/v0.4.0...v0.5.0)
|
39
|
+
|
40
|
+
**Merged pull requests:**
|
41
|
+
|
42
|
+
- Update development Ruby to 2.6.6 and gems [\#29](https://github.com/bensheldon/good_job/pull/29) ([bensheldon](https://github.com/bensheldon))
|
43
|
+
|
44
|
+
## [v0.4.0](https://github.com/bensheldon/good_job/tree/v0.4.0) (2020-03-31)
|
45
|
+
|
46
|
+
[Full Changelog](https://github.com/bensheldon/good_job/compare/v0.3.0...v0.4.0)
|
6
47
|
|
7
48
|
**Merged pull requests:**
|
8
49
|
|
@@ -18,6 +59,7 @@
|
|
18
59
|
- Update development Ruby to 2.6.5 [\#22](https://github.com/bensheldon/good_job/pull/22) ([bensheldon](https://github.com/bensheldon))
|
19
60
|
- Simplify the internal API, removing JobWrapper and InlineScheduler [\#21](https://github.com/bensheldon/good_job/pull/21) ([bensheldon](https://github.com/bensheldon))
|
20
61
|
- Generate a new future for every executed job [\#20](https://github.com/bensheldon/good_job/pull/20) ([bensheldon](https://github.com/bensheldon))
|
62
|
+
- Configuration for maximum number of job execution threads [\#18](https://github.com/bensheldon/good_job/pull/18) ([bensheldon](https://github.com/bensheldon))
|
21
63
|
|
22
64
|
## [v0.2.2](https://github.com/bensheldon/good_job/tree/v0.2.2) (2020-03-08)
|
23
65
|
|
@@ -25,7 +67,6 @@
|
|
25
67
|
|
26
68
|
**Merged pull requests:**
|
27
69
|
|
28
|
-
- Configuration for maximum number of job execution threads [\#18](https://github.com/bensheldon/good_job/pull/18) ([bensheldon](https://github.com/bensheldon))
|
29
70
|
- Gracefully shutdown Scheduler when executable receives TERM or INT [\#17](https://github.com/bensheldon/good_job/pull/17) ([bensheldon](https://github.com/bensheldon))
|
30
71
|
- Update Appraisals [\#16](https://github.com/bensheldon/good_job/pull/16) ([bensheldon](https://github.com/bensheldon))
|
31
72
|
|
data/README.md
CHANGED
@@ -1,25 +1,25 @@
|
|
1
1
|
# GoodJob
|
2
2
|
|
3
|
-
GoodJob is a multithreaded, Postgres-based ActiveJob backend for Ruby on Rails.
|
3
|
+
GoodJob is a multithreaded, Postgres-based, ActiveJob backend for Ruby on Rails.
|
4
4
|
|
5
|
-
Inspired by [Delayed::Job](https://github.com/collectiveidea/delayed_job) and [Que](https://github.com/que-rb/que), GoodJob
|
5
|
+
**Inspired by [Delayed::Job](https://github.com/collectiveidea/delayed_job) and [Que](https://github.com/que-rb/que), GoodJob is designed for maximum compatibility with Ruby on Rails, ActiveJob, and Postgres to be simple and performant for most workloads.**
|
6
6
|
|
7
|
-
-
|
8
|
-
-
|
9
|
-
-
|
10
|
-
-
|
7
|
+
- **Designed for ActiveJob.** Complete support for [async, queues, delays, priorities, timeouts, and retries](https://edgeguides.rubyonrails.org/active_job_basics.html) with near-zero configuration.
|
8
|
+
- **Built for Rails.** Fully adopts Ruby on Rails [threading and code execution guidelines](https://guides.rubyonrails.org/threading_and_code_execution.html) with [Concurrent::Ruby](https://github.com/ruby-concurrency/concurrent-ruby).
|
9
|
+
- **Backed by Postgres.** Relies upon Postgres integrity and session-level Advisory Locks to provide run-once safety and stay within the limits of `schema.rb`.
|
10
|
+
- **For most workloads.** Targets full-stack teams, economy-minded solo developers, and applications that enqueue less than 1-million jobs/day.
|
11
11
|
|
12
12
|
## Installation
|
13
13
|
|
14
14
|
Add this line to your application's Gemfile:
|
15
15
|
|
16
16
|
```ruby
|
17
|
-
gem 'good_job'
|
17
|
+
gem 'good_job'
|
18
18
|
```
|
19
19
|
|
20
20
|
And then execute:
|
21
21
|
```bash
|
22
|
-
$ bundle
|
22
|
+
$ bundle install
|
23
23
|
```
|
24
24
|
|
25
25
|
## Usage
|
@@ -43,6 +43,9 @@ $ bundle
|
|
43
43
|
t.integer :priority
|
44
44
|
t.jsonb :serialized_params
|
45
45
|
t.timestamp :scheduled_at
|
46
|
+
|
47
|
+
t.index :scheduled_at
|
48
|
+
t.index [:queue_name, :scheduled_at]
|
46
49
|
end
|
47
50
|
end
|
48
51
|
end
|
@@ -56,11 +59,26 @@ $ bundle
|
|
56
59
|
|
57
60
|
1. Configure the ActiveJob adapter:
|
58
61
|
```ruby
|
62
|
+
# config/application.rb
|
63
|
+
config.active_job.queue_adapter = :good_job
|
64
|
+
```
|
65
|
+
|
66
|
+
By default, using `:good_job` is equivalent to manually configuring the adapter:
|
67
|
+
|
68
|
+
```ruby
|
69
|
+
# config/environments/development.rb
|
70
|
+
config.active_job.queue_adapter = GoodJob::Adapter.new(execution_mode: :inline)
|
71
|
+
|
72
|
+
# config/environments/test.rb
|
73
|
+
config.active_job.queue_adapter = GoodJob::Adapter.new(execution_mode: :inline)
|
74
|
+
|
59
75
|
# config/environments/production.rb
|
60
|
-
config.active_job.queue_adapter = GoodJob::Adapter.new
|
76
|
+
config.active_job.queue_adapter = GoodJob::Adapter.new(execution_mode: :external)
|
77
|
+
```
|
61
78
|
|
62
|
-
|
63
|
-
|
79
|
+
1. Queue your job 🎉:
|
80
|
+
```ruby
|
81
|
+
YourJob.set(queue: :some_queue, wait: 5.minutes, priority: 10).perform_later
|
64
82
|
```
|
65
83
|
|
66
84
|
1. In production, the scheduler is designed to run in its own process:
|
@@ -68,6 +86,38 @@ $ bundle
|
|
68
86
|
$ bundle exec good_job
|
69
87
|
```
|
70
88
|
|
89
|
+
Configuration options available with `help`:
|
90
|
+
```bash
|
91
|
+
$ bundle exec good_job help start
|
92
|
+
|
93
|
+
# Usage:
|
94
|
+
# good_job start
|
95
|
+
#
|
96
|
+
# Options:
|
97
|
+
# [--max-threads=N] # Maximum number of threads to use for working jobs (default: ActiveRecord::Base.connection_pool.size)
|
98
|
+
# [--queues=queue1,queue2] # Queues to work from. Separate multiple queues with commas (default: *)
|
99
|
+
# [--poll-interval=N] # Interval between polls for available jobs in seconds (default: 1)
|
100
|
+
```
|
101
|
+
|
102
|
+
### Taking advantage of ActiveJob
|
103
|
+
|
104
|
+
ActiveJob has a rich set of built-in functionality for timeouts, error handling, and retrying. For example:
|
105
|
+
|
106
|
+
```ruby
|
107
|
+
class ApplicationJob < ActiveJob::Base
|
108
|
+
# Retry errors an infinite number of times with exponential back-off
|
109
|
+
retry_on StandardError, wait: :exponentially_longer, attempts: Float::INFINITY
|
110
|
+
|
111
|
+
# Timeout jobs after 10 minutes
|
112
|
+
JobTimeoutError = Class.new(StandardError)
|
113
|
+
around_perform do |_job, block|
|
114
|
+
Timeout.timeout(10.minutes, JobTimeoutError) do
|
115
|
+
block.call
|
116
|
+
end
|
117
|
+
end
|
118
|
+
end
|
119
|
+
```
|
120
|
+
|
71
121
|
### Configuring Job Execution Threads
|
72
122
|
|
73
123
|
GoodJob executes enqueued jobs using threads. There is a lot than can be said about [multithreaded behavior in Ruby on Rails](https://guides.rubyonrails.org/threading_and_code_execution.html), but briefly:
|
@@ -79,6 +129,29 @@ GoodJob executes enqueued jobs using threads. There is a lot than can be said ab
|
|
79
129
|
3. `$ RAILS_MAX_THREADS=4 bundle exec good_job`
|
80
130
|
4. Implicitly via Rails's database connection pool size (`ActiveRecord::Base.connection_pool.size`)
|
81
131
|
|
132
|
+
### Migrating to GoodJob from a different ActiveJob backend
|
133
|
+
|
134
|
+
If your application is already using an ActiveJob backend, you will need to install GoodJob to enqueue and perform newly created jobs _and_ finish performing pre-existing jobs on the previous backend.
|
135
|
+
|
136
|
+
1. Enqueue newly created jobs on GoodJob either entirely by setting `ActiveJob::Base.queue_adapter = :good_job` or progressively via individual job classes:
|
137
|
+
|
138
|
+
```ruby
|
139
|
+
# jobs/specific_job.rb
|
140
|
+
class SpecificJob < ApplicationJob
|
141
|
+
self.queue_adapter = :good_job
|
142
|
+
# ...
|
143
|
+
end
|
144
|
+
```
|
145
|
+
|
146
|
+
1. Continue running executors for both backends. For example, on Heroku it's possible to run [two processes](https://help.heroku.com/CTFS2TJK/how-do-i-run-multiple-processes-on-a-dyno) within the same dyno:
|
147
|
+
```procfile
|
148
|
+
# Procfile
|
149
|
+
# ...
|
150
|
+
worker: bundle exec que ./config/environment.rb & bundle exec good_job & wait -n
|
151
|
+
```
|
152
|
+
|
153
|
+
1. Once you are confident that no unperformed jobs remain in the previous ActiveJob backend, code and configuration for that backend can be completely removed.
|
154
|
+
|
82
155
|
## Development
|
83
156
|
|
84
157
|
To run tests:
|
@@ -94,6 +167,17 @@ $ bin/setup_test
|
|
94
167
|
$ bin/rspec
|
95
168
|
```
|
96
169
|
|
170
|
+
This gem uses Appraisal to run tests against multiple versions of Rails:
|
171
|
+
|
172
|
+
```bash
|
173
|
+
# Install Appraisal(s) gemfiles
|
174
|
+
$ bundle exec appraisal
|
175
|
+
|
176
|
+
# Run tests
|
177
|
+
$ bundle exec appraisal bin/rspec
|
178
|
+
|
179
|
+
```
|
180
|
+
|
97
181
|
For developing locally within another Ruby on Rails project:
|
98
182
|
|
99
183
|
```bash
|
@@ -115,7 +199,7 @@ Package maintainers can release this gem with the following [gem-release](https:
|
|
115
199
|
$ gem signin
|
116
200
|
|
117
201
|
# Update version number, changelog, and create git commit:
|
118
|
-
$ bundle exec rake commit_version
|
202
|
+
$ bundle exec rake commit_version[minor] # major,minor,patch
|
119
203
|
|
120
204
|
# ..and follow subsequent directions.
|
121
205
|
```
|
@@ -0,0 +1,19 @@
|
|
1
|
+
module ActiveJob
|
2
|
+
module QueueAdapters
|
3
|
+
class GoodJobAdapter < GoodJob::Adapter
|
4
|
+
def initialize(execution_mode: nil)
|
5
|
+
execution_mode = if execution_mode
|
6
|
+
execution_mode
|
7
|
+
elsif ENV['GOOD_JOB_EXECUTION_MODE'].present?
|
8
|
+
ENV['GOOD_JOB_EXECUTION_MODE'].to_sym
|
9
|
+
elsif Rails.env.development? || Rails.env.test?
|
10
|
+
:inline
|
11
|
+
else
|
12
|
+
:external
|
13
|
+
end
|
14
|
+
|
15
|
+
super(execution_mode: execution_mode)
|
16
|
+
end
|
17
|
+
end
|
18
|
+
end
|
19
|
+
end
|
data/lib/good_job.rb
CHANGED
data/lib/good_job/adapter.rb
CHANGED
@@ -1,7 +1,18 @@
|
|
1
1
|
module GoodJob
|
2
2
|
class Adapter
|
3
|
-
|
4
|
-
|
3
|
+
EXECUTION_MODES = [:inline, :external].freeze # TODO: async
|
4
|
+
|
5
|
+
def initialize(execution_mode: nil, inline: false)
|
6
|
+
if inline
|
7
|
+
ActiveSupport::Deprecation.warn('GoodJob::Adapter#new(inline: true) is deprecated; use GoodJob::Adapter.new(execution_mode: :inline) instead')
|
8
|
+
@execution_mode = :inline
|
9
|
+
elsif execution_mode
|
10
|
+
raise ArgumentError, "execution_mode: must be one of #{EXECUTION_MODES.join(', ')}." unless EXECUTION_MODES.include?(execution_mode)
|
11
|
+
|
12
|
+
@execution_mode = execution_mode
|
13
|
+
else
|
14
|
+
@execution_mode = :external
|
15
|
+
end
|
5
16
|
end
|
6
17
|
|
7
18
|
def enqueue(active_job)
|
@@ -11,13 +22,16 @@ module GoodJob
|
|
11
22
|
def enqueue_at(active_job, timestamp)
|
12
23
|
good_job = GoodJob::Job.enqueue(
|
13
24
|
active_job,
|
14
|
-
scheduled_at: timestamp ? Time.at(timestamp) : nil,
|
15
|
-
create_with_advisory_lock:
|
25
|
+
scheduled_at: timestamp ? Time.zone.at(timestamp) : nil,
|
26
|
+
create_with_advisory_lock: execute_inline?
|
16
27
|
)
|
17
28
|
|
18
|
-
if
|
19
|
-
|
20
|
-
|
29
|
+
if execute_inline?
|
30
|
+
begin
|
31
|
+
good_job.perform
|
32
|
+
ensure
|
33
|
+
good_job.advisory_unlock
|
34
|
+
end
|
21
35
|
end
|
22
36
|
|
23
37
|
good_job
|
@@ -27,10 +41,17 @@ module GoodJob
|
|
27
41
|
nil
|
28
42
|
end
|
29
43
|
|
30
|
-
|
44
|
+
def execute_inline?
|
45
|
+
@execution_mode == :inline
|
46
|
+
end
|
31
47
|
|
32
48
|
def inline?
|
33
|
-
|
49
|
+
ActiveSupport::Deprecation.warn('GoodJob::Adapter::inline? is deprecated; use GoodJob::Adapter::execute_inline? instead')
|
50
|
+
execute_inline?
|
51
|
+
end
|
52
|
+
|
53
|
+
def execute_externally?
|
54
|
+
@execution_mode == :external
|
34
55
|
end
|
35
56
|
end
|
36
57
|
end
|
data/lib/good_job/cli.rb
CHANGED
@@ -4,23 +4,59 @@ module GoodJob
|
|
4
4
|
class CLI < Thor
|
5
5
|
RAILS_ENVIRONMENT_RB = File.expand_path("config/environment.rb")
|
6
6
|
|
7
|
-
desc :start, "Start
|
8
|
-
method_option :max_threads,
|
7
|
+
desc :start, "Start job worker"
|
8
|
+
method_option :max_threads,
|
9
|
+
type: :numeric,
|
10
|
+
desc: "Maximum number of threads to use for working jobs (default: ActiveRecord::Base.connection_pool.size)"
|
11
|
+
method_option :queues,
|
12
|
+
type: :string,
|
13
|
+
banner: "queue1,queue2",
|
14
|
+
desc: "Queues to work from. Separate multiple queues with commas (default: *)"
|
15
|
+
method_option :poll_interval,
|
16
|
+
type: :numeric,
|
17
|
+
desc: "Interval between polls for available jobs in seconds (default: 1)"
|
9
18
|
def start
|
10
19
|
require RAILS_ENVIRONMENT_RB
|
11
20
|
|
12
|
-
max_threads =
|
13
|
-
|
14
|
-
|
15
|
-
|
21
|
+
max_threads = (
|
22
|
+
options[:max_threads] ||
|
23
|
+
ENV['GOOD_JOB_MAX_THREADS'] ||
|
24
|
+
ENV['RAILS_MAX_THREADS'] ||
|
25
|
+
ActiveRecord::Base.connection_pool.size
|
26
|
+
).to_i
|
16
27
|
|
17
|
-
|
18
|
-
|
28
|
+
queue_names = (
|
29
|
+
options[:queues] ||
|
30
|
+
ENV['GOOD_JOB_QUEUES'] ||
|
31
|
+
'*'
|
32
|
+
).split(',').map(&:strip)
|
19
33
|
|
34
|
+
poll_interval = (
|
35
|
+
options[:poll_interval] ||
|
36
|
+
ENV['GOOD_JOB_POLL_INTERVAL']
|
37
|
+
).to_i
|
38
|
+
|
39
|
+
job_query = GoodJob::Job.all.priority_ordered
|
40
|
+
queue_names_without_all = queue_names.reject { |q| q == '*' }
|
41
|
+
job_query = job_query.where(queue_name: queue_names_without_all) unless queue_names_without_all.size.zero?
|
42
|
+
|
43
|
+
job_performer = job_query.to_performer
|
44
|
+
|
45
|
+
$stdout.puts "GoodJob worker starting with max_threads=#{max_threads} on queues=#{queue_names.join(',')}"
|
46
|
+
|
47
|
+
timer_options = {}
|
48
|
+
timer_options[:execution_interval] = poll_interval if poll_interval.positive?
|
49
|
+
|
50
|
+
pool_options = {
|
51
|
+
max_threads: max_threads,
|
52
|
+
}
|
53
|
+
|
54
|
+
scheduler = GoodJob::Scheduler.new(job_performer, timer_options: timer_options, pool_options: pool_options)
|
55
|
+
|
56
|
+
@stop_good_job_executable = false
|
20
57
|
%w[INT TERM].each do |signal|
|
21
58
|
trap(signal) { @stop_good_job_executable = true }
|
22
59
|
end
|
23
|
-
@stop_good_job_executable = false
|
24
60
|
|
25
61
|
Kernel.loop do
|
26
62
|
sleep 0.1
|
data/lib/good_job/job.rb
CHANGED
@@ -2,16 +2,42 @@ module GoodJob
|
|
2
2
|
class Job < ActiveRecord::Base
|
3
3
|
include Lockable
|
4
4
|
|
5
|
-
|
5
|
+
DEFAULT_QUEUE_NAME = 'default'.freeze
|
6
|
+
DEFAULT_PRIORITY = 0
|
7
|
+
|
8
|
+
self.table_name = 'good_jobs'.freeze
|
9
|
+
|
10
|
+
scope :only_scheduled, -> { where(arel_table['scheduled_at'].lteq(Time.current)).or(where(scheduled_at: nil)) }
|
11
|
+
scope :priority_ordered, -> { order(priority: :desc) }
|
12
|
+
scope :to_performer, -> { Performer.new(self) }
|
13
|
+
|
14
|
+
class Performer
|
15
|
+
def initialize(query)
|
16
|
+
@query = query
|
17
|
+
end
|
18
|
+
|
19
|
+
def next
|
20
|
+
good_job = nil
|
21
|
+
|
22
|
+
@query.only_scheduled.limit(1).with_advisory_lock do |good_jobs|
|
23
|
+
good_job = good_jobs.first
|
24
|
+
break unless good_job
|
25
|
+
|
26
|
+
good_job.perform
|
27
|
+
end
|
28
|
+
|
29
|
+
good_job
|
30
|
+
end
|
31
|
+
end
|
6
32
|
|
7
33
|
def self.enqueue(active_job, scheduled_at: nil, create_with_advisory_lock: false)
|
8
34
|
good_job = nil
|
9
35
|
ActiveSupport::Notifications.instrument("enqueue_job.good_job", { active_job: active_job, scheduled_at: scheduled_at, create_with_advisory_lock: create_with_advisory_lock }) do |instrument_payload|
|
10
36
|
good_job = GoodJob::Job.new(
|
11
|
-
queue_name: active_job.queue_name,
|
12
|
-
priority: active_job.priority,
|
37
|
+
queue_name: active_job.queue_name.presence || DEFAULT_QUEUE_NAME,
|
38
|
+
priority: active_job.priority || DEFAULT_PRIORITY,
|
13
39
|
serialized_params: active_job.serialize,
|
14
|
-
scheduled_at: scheduled_at,
|
40
|
+
scheduled_at: scheduled_at || Time.current,
|
15
41
|
create_with_advisory_lock: create_with_advisory_lock
|
16
42
|
)
|
17
43
|
|
data/lib/good_job/lockable.rb
CHANGED
@@ -37,14 +37,17 @@ module GoodJob
|
|
37
37
|
scope :owns_advisory_locked, -> { joins_advisory_locks.where('"pg_locks"."pid" = pg_backend_pid()') }
|
38
38
|
|
39
39
|
attr_accessor :create_with_advisory_lock
|
40
|
+
|
40
41
|
after_create -> { advisory_lock }, if: :create_with_advisory_lock
|
41
42
|
end
|
42
43
|
|
43
44
|
class_methods do
|
44
|
-
def with_advisory_lock
|
45
|
+
def with_advisory_lock
|
46
|
+
raise ArgumentError, "Must provide a block" unless block_given?
|
47
|
+
|
45
48
|
records = advisory_lock.to_a
|
46
49
|
begin
|
47
|
-
|
50
|
+
yield(records)
|
48
51
|
ensure
|
49
52
|
records.each(&:advisory_unlock)
|
50
53
|
end
|
@@ -73,6 +76,8 @@ module GoodJob
|
|
73
76
|
end
|
74
77
|
|
75
78
|
def with_advisory_lock
|
79
|
+
raise ArgumentError, "Must provide a block" unless block_given?
|
80
|
+
|
76
81
|
advisory_lock!
|
77
82
|
yield
|
78
83
|
ensure
|
data/lib/good_job/scheduler.rb
CHANGED
@@ -20,9 +20,10 @@ module GoodJob
|
|
20
20
|
fallback_policy: :abort, # shouldn't matter -- 0 max queue
|
21
21
|
}.freeze
|
22
22
|
|
23
|
-
def initialize(
|
24
|
-
|
23
|
+
def initialize(performer, timer_options: {}, pool_options: {})
|
24
|
+
raise ArgumentError, "Performer argument must implement #next" unless performer.respond_to?(:next)
|
25
25
|
|
26
|
+
@performer = performer
|
26
27
|
@pool = Concurrent::ThreadPoolExecutor.new(DEFAULT_POOL_OPTIONS.merge(pool_options))
|
27
28
|
@timer = Concurrent::TimerTask.new(DEFAULT_TIMER_OPTIONS.merge(timer_options)) do
|
28
29
|
idle_threads = @pool.max_length - @pool.length
|
@@ -32,10 +33,6 @@ module GoodJob
|
|
32
33
|
@timer.execute
|
33
34
|
end
|
34
35
|
|
35
|
-
def ordered_query
|
36
|
-
@query.where("scheduled_at < ?", Time.current).or(@query.where(scheduled_at: nil)).order(priority: :desc)
|
37
|
-
end
|
38
|
-
|
39
36
|
def execute
|
40
37
|
end
|
41
38
|
|
@@ -61,19 +58,10 @@ module GoodJob
|
|
61
58
|
end
|
62
59
|
|
63
60
|
def create_thread
|
64
|
-
future = Concurrent::Future.new(args: [
|
65
|
-
|
66
|
-
|
67
|
-
|
68
|
-
query.limit(1).with_advisory_lock do |good_jobs|
|
69
|
-
good_job = good_jobs.first
|
70
|
-
break unless good_job
|
71
|
-
|
72
|
-
good_job.perform
|
73
|
-
end
|
74
|
-
end
|
75
|
-
|
76
|
-
good_job
|
61
|
+
future = Concurrent::Future.new(args: [@performer], executor: @pool) do |performer|
|
62
|
+
result = nil
|
63
|
+
Rails.application.executor.wrap { result = performer.next }
|
64
|
+
result
|
77
65
|
end
|
78
66
|
future.add_observer(self, :task_observer)
|
79
67
|
future.execute
|
@@ -83,9 +71,9 @@ module GoodJob
|
|
83
71
|
ActiveSupport::Notifications.instrument("finished_timer_task.good_job", { result: executed_task, error: error, time: time })
|
84
72
|
end
|
85
73
|
|
86
|
-
def task_observer(time,
|
87
|
-
ActiveSupport::Notifications.instrument("finished_job_task.good_job", {
|
88
|
-
create_thread if
|
74
|
+
def task_observer(time, result, error)
|
75
|
+
ActiveSupport::Notifications.instrument("finished_job_task.good_job", { result: result, error: error, time: time })
|
76
|
+
create_thread if result
|
89
77
|
end
|
90
78
|
end
|
91
79
|
end
|
data/lib/good_job/version.rb
CHANGED
metadata
CHANGED
@@ -1,14 +1,14 @@
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
2
2
|
name: good_job
|
3
3
|
version: !ruby/object:Gem::Version
|
4
|
-
version: 0.
|
4
|
+
version: 0.8.1
|
5
5
|
platform: ruby
|
6
6
|
authors:
|
7
7
|
- Ben Sheldon
|
8
8
|
autorequire:
|
9
9
|
bindir: exe
|
10
10
|
cert_chain: []
|
11
|
-
date: 2020-
|
11
|
+
date: 2020-07-18 00:00:00.000000000 Z
|
12
12
|
dependencies:
|
13
13
|
- !ruby/object:Gem::Dependency
|
14
14
|
name: concurrent-ruby
|
@@ -178,6 +178,34 @@ dependencies:
|
|
178
178
|
- - ">="
|
179
179
|
- !ruby/object:Gem::Version
|
180
180
|
version: '0'
|
181
|
+
- !ruby/object:Gem::Dependency
|
182
|
+
name: rubocop-performance
|
183
|
+
requirement: !ruby/object:Gem::Requirement
|
184
|
+
requirements:
|
185
|
+
- - ">="
|
186
|
+
- !ruby/object:Gem::Version
|
187
|
+
version: '0'
|
188
|
+
type: :development
|
189
|
+
prerelease: false
|
190
|
+
version_requirements: !ruby/object:Gem::Requirement
|
191
|
+
requirements:
|
192
|
+
- - ">="
|
193
|
+
- !ruby/object:Gem::Version
|
194
|
+
version: '0'
|
195
|
+
- !ruby/object:Gem::Dependency
|
196
|
+
name: rubocop-rails
|
197
|
+
requirement: !ruby/object:Gem::Requirement
|
198
|
+
requirements:
|
199
|
+
- - ">="
|
200
|
+
- !ruby/object:Gem::Version
|
201
|
+
version: '0'
|
202
|
+
type: :development
|
203
|
+
prerelease: false
|
204
|
+
version_requirements: !ruby/object:Gem::Requirement
|
205
|
+
requirements:
|
206
|
+
- - ">="
|
207
|
+
- !ruby/object:Gem::Version
|
208
|
+
version: '0'
|
181
209
|
- !ruby/object:Gem::Dependency
|
182
210
|
name: rubocop-rspec
|
183
211
|
requirement: !ruby/object:Gem::Requirement
|
@@ -207,6 +235,7 @@ files:
|
|
207
235
|
- LICENSE.txt
|
208
236
|
- README.md
|
209
237
|
- exe/good_job
|
238
|
+
- lib/active_job/queue_adapters/good_job_adapter.rb
|
210
239
|
- lib/good_job.rb
|
211
240
|
- lib/good_job/adapter.rb
|
212
241
|
- lib/good_job/cli.rb
|