good_job 0.8.2 → 1.0.3

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: 6c630bdd77afa672158924fa33dd200506354db2df4c1e795c85b3e02becc71b
4
- data.tar.gz: 39ac5362e8832e62f50ba9703060af5907dcbacd64f8765b9cafaa98f10dbbb6
3
+ metadata.gz: fc7b3997aaa38583efb4c91e66a2a5f9b8bfd75d4c14c9baaf8f1af7c236a42e
4
+ data.tar.gz: 285b441a5e5add440142960884370a1b74168266252fa48ce569cbb56e80f240
5
5
  SHA512:
6
- metadata.gz: 86a6d5613d68428276244373b2c718a327538fe30c0461b1eaec0156ca51f72b2863d6e268e96e59f4c40e8096059c05758cc1ce83d02b680916f09a57bff428
7
- data.tar.gz: 50749d69aa79ea96e3a9081b2a0cd02bdbeb4537d3da0bba748ffc3c43fb251149f84ea8cc1620fbc211794ab499fc713de8d57f8b74f49cead98fb1b8d4c1bd
6
+ metadata.gz: f54e83d6ac19d80da2123a395ccf6ffd7eb2faab8e68bd5d50a572c8f0bf64fbc907125a47b7ff91dccac5497af113d043bba4f62ee6ae10fecaf3c004a1114d
7
+ data.tar.gz: 0b65a60bde26db0cfcfa55f166e80043bcbccb6e2c80f5de1840d2deae3ee28880c904bb53ffc2478ad834ffef8cfe567cee9a179932e20e4f92fdf8504838e4
@@ -1,20 +1,98 @@
1
1
  # Changelog
2
2
 
3
+ ## [v1.0.3](https://github.com/bensheldon/good_job/tree/v1.0.3) (2020-07-25)
4
+
5
+ [Full Changelog](https://github.com/bensheldon/good_job/compare/v1.0.2...v1.0.3)
6
+
7
+ **Fixed bugs:**
8
+
9
+ - Preserve GoodJob::Jobs when a StandardError is raised [\#60](https://github.com/bensheldon/good_job/issues/60)
10
+
11
+ **Closed issues:**
12
+
13
+ - Have an initial setup generator [\#6](https://github.com/bensheldon/good_job/issues/6)
14
+
15
+ **Merged pull requests:**
16
+
17
+ - Re-perform a job if a StandardError bubbles up; better document job reliability [\#62](https://github.com/bensheldon/good_job/pull/62) ([bensheldon](https://github.com/bensheldon))
18
+ - Update the setup documentation to use correct bin setup command [\#61](https://github.com/bensheldon/good_job/pull/61) ([jm96441n](https://github.com/jm96441n))
19
+
20
+ ## [v1.0.2](https://github.com/bensheldon/good_job/tree/v1.0.2) (2020-07-25)
21
+
22
+ [Full Changelog](https://github.com/bensheldon/good_job/compare/v1.0.1...v1.0.2)
23
+
24
+ **Fixed bugs:**
25
+
26
+ - Fix counting of available execution threads [\#58](https://github.com/bensheldon/good_job/pull/58) ([bensheldon](https://github.com/bensheldon))
27
+
28
+ **Closed issues:**
29
+
30
+ - repeating/recurring jobs [\#53](https://github.com/bensheldon/good_job/issues/53)
31
+
32
+ **Merged pull requests:**
33
+
34
+ - Add migration generator [\#56](https://github.com/bensheldon/good_job/pull/56) ([thedanbob](https://github.com/thedanbob))
35
+ - Fix migration script in readme [\#55](https://github.com/bensheldon/good_job/pull/55) ([thedanbob](https://github.com/thedanbob))
36
+
37
+ ## [v1.0.1](https://github.com/bensheldon/good_job/tree/v1.0.1) (2020-07-22)
38
+
39
+ [Full Changelog](https://github.com/bensheldon/good_job/compare/v1.0.0...v1.0.1)
40
+
41
+ **Merged pull requests:**
42
+
43
+ - Change threadpool idletime default to 60 seconds from 0 [\#49](https://github.com/bensheldon/good_job/pull/49) ([bensheldon](https://github.com/bensheldon))
44
+
45
+ ## [v1.0.0](https://github.com/bensheldon/good_job/tree/v1.0.0) (2020-07-20)
46
+
47
+ [Full Changelog](https://github.com/bensheldon/good_job/compare/v0.9.0...v1.0.0)
48
+
49
+ ## [v0.9.0](https://github.com/bensheldon/good_job/tree/v0.9.0) (2020-07-20)
50
+
51
+ [Full Changelog](https://github.com/bensheldon/good_job/compare/v0.8.2...v0.9.0)
52
+
53
+ **Merged pull requests:**
54
+
55
+ - Allow preservation of finished job records [\#46](https://github.com/bensheldon/good_job/pull/46) ([bensheldon](https://github.com/bensheldon))
56
+
3
57
  ## [v0.8.2](https://github.com/bensheldon/good_job/tree/v0.8.2) (2020-07-18)
4
58
 
5
- [Full Changelog](https://github.com/bensheldon/good_job/compare/v0.6.0...v0.8.2)
59
+ [Full Changelog](https://github.com/bensheldon/good_job/compare/v0.8.1...v0.8.2)
6
60
 
7
61
  **Closed issues:**
8
62
 
9
- - Always store a default priority \(0\) and scheduled\_at\(Time.current\) [\#30](https://github.com/bensheldon/good_job/issues/30)
10
63
  - Add a job timeout configuration to time out jobs that have run too long [\#19](https://github.com/bensheldon/good_job/issues/19)
11
64
 
12
65
  **Merged pull requests:**
13
66
 
14
67
  - Run Github Action tests on PRs from forks [\#44](https://github.com/bensheldon/good_job/pull/44) ([bensheldon](https://github.com/bensheldon))
15
68
  - Fix Rubygems homepage URL [\#43](https://github.com/bensheldon/good_job/pull/43) ([joshmn](https://github.com/joshmn))
69
+
70
+ ## [v0.8.1](https://github.com/bensheldon/good_job/tree/v0.8.1) (2020-07-18)
71
+
72
+ [Full Changelog](https://github.com/bensheldon/good_job/compare/v0.8.0...v0.8.1)
73
+
74
+ **Merged pull requests:**
75
+
16
76
  - Move where\(scheduled\_at: Time.current\) into dynamic part of GoodJob::Job::Performer [\#42](https://github.com/bensheldon/good_job/pull/42) ([bensheldon](https://github.com/bensheldon))
77
+
78
+ ## [v0.8.0](https://github.com/bensheldon/good_job/tree/v0.8.0) (2020-07-17)
79
+
80
+ [Full Changelog](https://github.com/bensheldon/good_job/compare/v0.7.0...v0.8.0)
81
+
82
+ **Merged pull requests:**
83
+
17
84
  - Replace Adapter inline boolean kwarg with execution\_mode instead [\#41](https://github.com/bensheldon/good_job/pull/41) ([bensheldon](https://github.com/bensheldon))
85
+
86
+ ## [v0.7.0](https://github.com/bensheldon/good_job/tree/v0.7.0) (2020-07-16)
87
+
88
+ [Full Changelog](https://github.com/bensheldon/good_job/compare/v0.6.0...v0.7.0)
89
+
90
+ **Closed issues:**
91
+
92
+ - Always store a default priority \(0\) and scheduled\_at\(Time.current\) [\#30](https://github.com/bensheldon/good_job/issues/30)
93
+
94
+ **Merged pull requests:**
95
+
18
96
  - Add more examples to Readme [\#39](https://github.com/bensheldon/good_job/pull/39) ([bensheldon](https://github.com/bensheldon))
19
97
  - Add additional Rubocops and lint [\#38](https://github.com/bensheldon/good_job/pull/38) ([bensheldon](https://github.com/bensheldon))
20
98
  - Always store a default queue\_name, priority and scheduled\_at; index by queue\_name and scheduled\_at [\#37](https://github.com/bensheldon/good_job/pull/37) ([bensheldon](https://github.com/bensheldon))
@@ -35,6 +113,7 @@
35
113
  - Add configuration options to good\_job executable [\#33](https://github.com/bensheldon/good_job/pull/33) ([bensheldon](https://github.com/bensheldon))
36
114
  - Extract Job querying behavior out of Scheduler [\#31](https://github.com/bensheldon/good_job/pull/31) ([bensheldon](https://github.com/bensheldon))
37
115
  - Allow configuration of Rails queue adapter with `:good\_job` [\#28](https://github.com/bensheldon/good_job/pull/28) ([bensheldon](https://github.com/bensheldon))
116
+ - Update development Ruby to 2.6.5 [\#22](https://github.com/bensheldon/good_job/pull/22) ([bensheldon](https://github.com/bensheldon))
38
117
 
39
118
  ## [v0.5.0](https://github.com/bensheldon/good_job/tree/v0.5.0) (2020-07-13)
40
119
 
@@ -59,7 +138,6 @@
59
138
 
60
139
  **Merged pull requests:**
61
140
 
62
- - Update development Ruby to 2.6.5 [\#22](https://github.com/bensheldon/good_job/pull/22) ([bensheldon](https://github.com/bensheldon))
63
141
  - Simplify the internal API, removing JobWrapper and InlineScheduler [\#21](https://github.com/bensheldon/good_job/pull/21) ([bensheldon](https://github.com/bensheldon))
64
142
  - Generate a new future for every executed job [\#20](https://github.com/bensheldon/good_job/pull/20) ([bensheldon](https://github.com/bensheldon))
65
143
  - Configuration for maximum number of job execution threads [\#18](https://github.com/bensheldon/good_job/pull/18) ([bensheldon](https://github.com/bensheldon))
data/README.md CHANGED
@@ -9,6 +9,8 @@ GoodJob is a multithreaded, Postgres-based, ActiveJob backend for Ruby on Rails.
9
9
  - **Backed by Postgres.** Relies upon Postgres integrity and session-level Advisory Locks to provide run-once safety and stay within the limits of `schema.rb`.
10
10
  - **For most workloads.** Targets full-stack teams, economy-minded solo developers, and applications that enqueue less than 1-million jobs/day.
11
11
 
12
+ For more of the story of GoodJob, read the [introductory blog post](https://island94.org/2020/07/introducing-goodjob-1-0).
13
+
12
14
  ## Installation
13
15
 
14
16
  Add this line to your application's Gemfile:
@@ -26,31 +28,9 @@ $ bundle install
26
28
 
27
29
  1. Create a database migration:
28
30
  ```bash
29
- $ bin/rails g migration CreateGoodJobs
31
+ $ bin/rails g good_job:install
30
32
  ```
31
33
 
32
- Add to the newly created migration file:
33
-
34
- ```ruby
35
- class CreateGoodJobs < ActiveRecord::Migration[6.0]
36
- def change
37
- enable_extension 'pgcrypto'
38
-
39
- create_table :good_jobs, id: :uuid do |t|
40
- t.timestamps
41
-
42
- t.text :queue_name
43
- t.integer :priority
44
- t.jsonb :serialized_params
45
- t.timestamp :scheduled_at
46
-
47
- t.index :scheduled_at
48
- t.index [:queue_name, :scheduled_at]
49
- end
50
- end
51
- end
52
- ```
53
-
54
34
  Run the migration:
55
35
 
56
36
  ```bash
@@ -99,25 +79,85 @@ $ bundle install
99
79
  # [--poll-interval=N] # Interval between polls for available jobs in seconds (default: 1)
100
80
  ```
101
81
 
102
- ### Taking advantage of ActiveJob
82
+ ### Error handling, retries, and reliability
83
+
84
+ GoodJob guarantees _at-least-once_ performance of jobs. GoodJob fully supports ActiveJob's built-in functionality for error handling, retries and timeouts.
85
+
86
+ #### Error handling
87
+
88
+ By default, if a job raises an error while it is being performed, _and it bubbles up to the GoodJob backend_, GoodJob will be immediately re-perform the job until it finishes successfully.
103
89
 
104
- ActiveJob has a rich set of built-in functionality for timeouts, error handling, and retrying. For example:
90
+ - `Exception`-type errors, such as a SIGINT, will always cause a job to be re-performed.
91
+ - `StandardError`-type errors, by default, will cause a job to be re-performed, though this is configurable:
92
+
93
+ ```ruby
94
+ # config/initializers/good_job.rb
95
+ GoodJob.reperform_jobs_on_standard_error = true # => default
96
+ ```
97
+
98
+ ### Retrying jobs
99
+
100
+ ActiveJob can be configured to retry an infinite number of times, with an exponential backoff. Using ActiveJob's `retry_on` will ensure that errors do not bubble up to the GoodJob backend:
105
101
 
106
102
  ```ruby
107
103
  class ApplicationJob < ActiveJob::Base
108
- # Retry errors an infinite number of times with exponential back-off
109
104
  retry_on StandardError, wait: :exponentially_longer, attempts: Float::INFINITY
105
+ # ...
106
+ end
107
+ ```
108
+
109
+ When specifying a limited number of retries, care must be taken to ensure that an error does not bubble up to the GoodJob backend because that will result in the job being re-performed:
110
+
111
+ ```ruby
112
+ class ApplicationJob < ActiveJob::Base
113
+ retry_on StandardError, attempts: 5 do |_job, _exception|
114
+ # Log error, etc.
115
+ # You must implement this block, otherwise,
116
+ # Active Job will re-raise the error.
117
+ # Do not re-raise the error, otherwise
118
+ # GoodJob will immediately re-perform the job.
119
+ end
120
+ # ...
121
+ end
122
+ ```
110
123
 
111
- # Timeout jobs after 10 minutes
124
+ GoodJob can be configured to allow omitting `retry_on`'s block argument and implicitly discard un-handled errors:
125
+
126
+ ```ruby
127
+ # config/initializers/good_job.rb
128
+
129
+ # Do NOT re-perform a job if a StandardError bubbles up to the GoodJob backend
130
+ GoodJob.reperform_jobs_on_standard_error = false
131
+ ```
132
+
133
+ ActiveJob's `discard_on` functionality is supported too.
134
+
135
+ #### ActionMailer retries
136
+
137
+ Using a Mailer's `#deliver_later` will enqueue an instance of `ActionMailer::DeliveryJob` which inherits from `ActiveJob::Base` rather than your applications `ApplicationJob`. You can use an initializer to configure retries on `ActionMailer::DeliveryJob`:
138
+
139
+ ```ruby
140
+ # config/initializers/good_job.rb
141
+ ActionMailer::DeliveryJob.retry_on StandardError, wait: :exponentially_longer, attempts: Float::INFINITY
142
+ ```
143
+
144
+ #### Timeouts
145
+
146
+ Job timeouts can be configured with an `around_perform`:
147
+
148
+ ```ruby
149
+ class ApplicationJob < ActiveJob::Base
112
150
  JobTimeoutError = Class.new(StandardError)
151
+
113
152
  around_perform do |_job, block|
153
+ # Timeout jobs after 10 minutes
114
154
  Timeout.timeout(10.minutes, JobTimeoutError) do
115
155
  block.call
116
156
  end
117
157
  end
118
158
  end
119
159
  ```
120
-
160
+
121
161
  ### Configuring Job Execution Threads
122
162
 
123
163
  GoodJob executes enqueued jobs using threads. There is a lot than can be said about [multithreaded behavior in Ruby on Rails](https://guides.rubyonrails.org/threading_and_code_execution.html), but briefly:
@@ -152,6 +192,32 @@ If your application is already using an ActiveJob backend, you will need to inst
152
192
 
153
193
  1. Once you are confident that no unperformed jobs remain in the previous ActiveJob backend, code and configuration for that backend can be completely removed.
154
194
 
195
+ ### Monitoring and preserving worked jobs
196
+
197
+ GoodJob is fully instrumented with [`ActiveSupport::Notifications`](https://edgeguides.rubyonrails.org/active_support_instrumentation.html#introduction-to-instrumentation).
198
+
199
+ By default, GoodJob will delete job records after they are run, regardless of whether they succeed or not (raising a kind of `StandardError`), unless they are interrupted (raising a kind of `Exception`).
200
+
201
+ To preserve job records for later inspection, set an initializer:
202
+
203
+ ```ruby
204
+ # config/initializers/good_job.rb
205
+ GoodJob.preserve_job_records = true
206
+ ```
207
+
208
+ It is also necessary to delete these preserved jobs from the database after a certain time period:
209
+
210
+ - For example, in a Rake task:
211
+
212
+ ```ruby
213
+ # GoodJob::Job.finished(1.day.ago).delete_all
214
+ ```
215
+ - For example, using the `good_job` command-line utility:
216
+
217
+ ```bash
218
+ $ bundle exec good_job cleanup_preserved_jobs --before-seconds-ago=86400
219
+ ```
220
+
155
221
  ## Development
156
222
 
157
223
  To run tests:
@@ -161,7 +227,7 @@ To run tests:
161
227
  $ git clone git@github.com:bensheldon/good_job.git
162
228
 
163
229
  # Set up the local environment
164
- $ bin/setup_test
230
+ $ bin/setup
165
231
 
166
232
  # Run the tests
167
233
  $ bin/rspec
@@ -0,0 +1,24 @@
1
+ require 'rails/generators'
2
+ require 'rails/generators/active_record'
3
+
4
+ module GoodJob
5
+ class InstallGenerator < Rails::Generators::Base
6
+ include Rails::Generators::Migration
7
+
8
+ class << self
9
+ delegate :next_migration_number, to: ActiveRecord::Generators::Base
10
+ end
11
+
12
+ source_paths << File.join(File.dirname(__FILE__), "templates")
13
+
14
+ def create_migration_file
15
+ migration_template 'migration.rb.erb', 'db/migrate/create_good_jobs.rb', migration_version: migration_version
16
+ end
17
+
18
+ private
19
+
20
+ def migration_version
21
+ "[#{ActiveRecord::VERSION::MAJOR}.#{ActiveRecord::VERSION::MINOR}]"
22
+ end
23
+ end
24
+ end
@@ -0,0 +1,20 @@
1
+ class CreateGoodJobs < ActiveRecord::Migration<%= migration_version %>
2
+ def change
3
+ enable_extension 'pgcrypto'
4
+
5
+ create_table :good_jobs, id: :uuid do |t|
6
+ t.text :queue_name
7
+ t.integer :priority
8
+ t.jsonb :serialized_params
9
+ t.timestamp :scheduled_at
10
+ t.timestamp :performed_at
11
+ t.timestamp :finished_at
12
+ t.text :error
13
+
14
+ t.timestamps
15
+ end
16
+
17
+ add_index :good_jobs, :scheduled_at, where: "(finished_at IS NULL)"
18
+ add_index :good_jobs, [:queue_name, :scheduled_at], where: "(finished_at IS NULL)"
19
+ end
20
+ end
@@ -4,13 +4,16 @@ require 'good_job/railtie'
4
4
  require 'good_job/logging'
5
5
  require 'good_job/lockable'
6
6
  require 'good_job/job'
7
- require "good_job/scheduler"
7
+ require 'good_job/scheduler'
8
8
  require 'good_job/adapter'
9
9
  require 'good_job/pg_locks'
10
+ require 'good_job/performer'
10
11
 
11
12
  require 'active_job/queue_adapters/good_job_adapter'
12
13
 
13
14
  module GoodJob
15
+ mattr_accessor :preserve_job_records, default: false
16
+ mattr_accessor :reperform_jobs_on_standard_error, default: true
14
17
  include Logging
15
18
 
16
19
  ActiveSupport.run_load_hooks(:good_job, self)
@@ -39,8 +39,7 @@ module GoodJob
39
39
  job_query = GoodJob::Job.all.priority_ordered
40
40
  queue_names_without_all = queue_names.reject { |q| q == '*' }
41
41
  job_query = job_query.where(queue_name: queue_names_without_all) unless queue_names_without_all.size.zero?
42
-
43
- job_performer = job_query.to_performer
42
+ job_performer = GoodJob::Performer.new(job_query, :perform_with_advisory_lock)
44
43
 
45
44
  $stdout.puts "GoodJob worker starting with max_threads=#{max_threads} on queues=#{queue_names.join(',')}"
46
45
 
@@ -68,6 +67,19 @@ module GoodJob
68
67
  $stdout.puts "GoodJob's jobs finished, exiting..."
69
68
  end
70
69
 
70
+ desc :cleanup_preserved_jobs, "Delete preserved job records"
71
+ method_option :before_seconds_ago,
72
+ type: :numeric,
73
+ default: 24 * 60 * 60,
74
+ desc: "Delete records finished more than this many seconds ago"
75
+ def cleanup_preserved_jobs
76
+ require RAILS_ENVIRONMENT_RB
77
+
78
+ timestamp = Time.current - options[:before_seconds_ago]
79
+ result = GoodJob::Job.finished(timestamp).delete_all
80
+ $stdout.puts "Deleted #{result} preserved #{'job'.pluralize(result)} finished before #{timestamp}."
81
+ end
82
+
71
83
  default_task :start
72
84
  end
73
85
  end
@@ -2,32 +2,38 @@ module GoodJob
2
2
  class Job < ActiveRecord::Base
3
3
  include Lockable
4
4
 
5
+ PreviouslyPerformedError = Class.new(StandardError)
6
+
5
7
  DEFAULT_QUEUE_NAME = 'default'.freeze
6
8
  DEFAULT_PRIORITY = 0
7
9
 
8
10
  self.table_name = 'good_jobs'.freeze
9
11
 
12
+ scope :unfinished, (lambda do
13
+ if column_names.include?('finished_at')
14
+ where(finished_at: nil)
15
+ else
16
+ ActiveSupport::Deprecation.warn('GoodJob expects a good_jobs.finished_at column to exist. Please see the GoodJob README.md for migration instructions.')
17
+ nil
18
+ end
19
+ end)
10
20
  scope :only_scheduled, -> { where(arel_table['scheduled_at'].lteq(Time.current)).or(where(scheduled_at: nil)) }
11
21
  scope :priority_ordered, -> { order(priority: :desc) }
12
- scope :to_performer, -> { Performer.new(self) }
13
-
14
- class Performer
15
- def initialize(query)
16
- @query = query
17
- end
22
+ scope :finished, ->(timestamp = nil) { timestamp ? where(arel_table['finished_at'].lteq(timestamp)) : where.not(finished_at: nil) }
18
23
 
19
- def next
20
- good_job = nil
24
+ def self.perform_with_advisory_lock
25
+ good_job = nil
26
+ result = nil
27
+ error = nil
21
28
 
22
- @query.only_scheduled.limit(1).with_advisory_lock do |good_jobs|
23
- good_job = good_jobs.first
24
- break unless good_job
29
+ unfinished.only_scheduled.limit(1).with_advisory_lock do |good_jobs|
30
+ good_job = good_jobs.first
31
+ break unless good_job
25
32
 
26
- good_job.perform
27
- end
28
-
29
- good_job
33
+ result, error = good_job.perform
30
34
  end
35
+
36
+ [good_job, result, error] if good_job
31
37
  end
32
38
 
33
39
  def self.enqueue(active_job, scheduled_at: nil, create_with_advisory_lock: false)
@@ -50,16 +56,52 @@ module GoodJob
50
56
  good_job
51
57
  end
52
58
 
53
- def perform
59
+ def perform(destroy_after: !GoodJob.preserve_job_records, reperform_on_standard_error: GoodJob.reperform_jobs_on_standard_error)
60
+ raise PreviouslyPerformedError, 'Cannot perform a job that has already been performed' if finished_at
61
+
62
+ result = nil
63
+ rescued_error = nil
64
+ error = nil
65
+
54
66
  ActiveSupport::Notifications.instrument("before_perform_job.good_job", { good_job: self })
55
- ActiveSupport::Notifications.instrument("perform_job.good_job", { good_job: self }) do
56
- params = serialized_params.merge(
57
- "provider_job_id" => id
58
- )
59
- ActiveJob::Base.execute(params)
67
+ self.performed_at = Time.current
68
+ save! unless destroy_after
60
69
 
61
- destroy!
70
+ params = serialized_params.merge(
71
+ "provider_job_id" => id
72
+ )
73
+
74
+ begin
75
+ ActiveSupport::Notifications.instrument("perform_job.good_job", { good_job: self }) do
76
+ result = ActiveJob::Base.execute(params)
77
+ end
78
+ rescue StandardError => e
79
+ rescued_error = e
80
+ end
81
+
82
+ if rescued_error
83
+ error = rescued_error
84
+ elsif result.is_a?(Exception)
85
+ error = result
86
+ result = nil
87
+ end
88
+
89
+ error_message = "#{error.class}: #{error.message}" if error
90
+ self.error = error_message
91
+
92
+ if rescued_error && reperform_on_standard_error
93
+ save!
94
+ else
95
+ self.finished_at = Time.current
96
+
97
+ if destroy_after
98
+ destroy!
99
+ else
100
+ save!
101
+ end
62
102
  end
103
+
104
+ [result, error]
63
105
  end
64
106
  end
65
107
  end
@@ -0,0 +1,12 @@
1
+ module GoodJob
2
+ class Performer
3
+ def initialize(target, method_name)
4
+ @target = target
5
+ @method_name = method_name
6
+ end
7
+
8
+ def next
9
+ @target.public_send(@method_name)
10
+ end
11
+ end
12
+ end
@@ -15,19 +15,18 @@ module GoodJob
15
15
  min_threads: 0,
16
16
  max_threads: Concurrent.processor_count,
17
17
  auto_terminate: true,
18
- idletime: 0,
19
- max_queue: 0,
20
- fallback_policy: :abort, # shouldn't matter -- 0 max queue
18
+ idletime: 60,
19
+ max_queue: -1,
20
+ fallback_policy: :discard,
21
21
  }.freeze
22
22
 
23
23
  def initialize(performer, timer_options: {}, pool_options: {})
24
24
  raise ArgumentError, "Performer argument must implement #next" unless performer.respond_to?(:next)
25
25
 
26
26
  @performer = performer
27
- @pool = Concurrent::ThreadPoolExecutor.new(DEFAULT_POOL_OPTIONS.merge(pool_options))
27
+ @pool = ThreadPoolExecutor.new(DEFAULT_POOL_OPTIONS.merge(pool_options))
28
28
  @timer = Concurrent::TimerTask.new(DEFAULT_TIMER_OPTIONS.merge(timer_options)) do
29
- idle_threads = @pool.max_length - @pool.length
30
- create_thread if idle_threads.positive?
29
+ create_thread
31
30
  end
32
31
  @timer.add_observer(self, :timer_observer)
33
32
  @timer.execute
@@ -58,22 +57,36 @@ module GoodJob
58
57
  end
59
58
 
60
59
  def create_thread
60
+ return false unless @pool.ready_worker_count.positive?
61
+
61
62
  future = Concurrent::Future.new(args: [@performer], executor: @pool) do |performer|
62
- result = nil
63
- Rails.application.executor.wrap { result = performer.next }
64
- result
63
+ output = nil
64
+ Rails.application.executor.wrap { output = performer.next }
65
+ output
65
66
  end
66
67
  future.add_observer(self, :task_observer)
67
68
  future.execute
68
69
  end
69
70
 
70
- def timer_observer(time, executed_task, error)
71
- ActiveSupport::Notifications.instrument("finished_timer_task.good_job", { result: executed_task, error: error, time: time })
71
+ def timer_observer(time, executed_task, thread_error)
72
+ ActiveSupport::Notifications.instrument("finished_timer_task.good_job", { result: executed_task, error: thread_error, time: time })
73
+ end
74
+
75
+ def task_observer(time, output, thread_error)
76
+ ActiveSupport::Notifications.instrument("finished_job_task.good_job", { result: output, error: thread_error, time: time })
77
+ create_thread if output
72
78
  end
73
79
 
74
- def task_observer(time, result, error)
75
- ActiveSupport::Notifications.instrument("finished_job_task.good_job", { result: result, error: error, time: time })
76
- create_thread if result
80
+ class ThreadPoolExecutor < Concurrent::ThreadPoolExecutor
81
+ # https://github.com/ruby-concurrency/concurrent-ruby/issues/684#issuecomment-427594437
82
+ def ready_worker_count
83
+ synchronize do
84
+ workers_still_to_be_created = @max_length - @pool.length
85
+ workers_created_but_waiting = @ready.length
86
+
87
+ workers_still_to_be_created + workers_created_but_waiting
88
+ end
89
+ end
77
90
  end
78
91
  end
79
92
  end
@@ -1,3 +1,3 @@
1
1
  module GoodJob
2
- VERSION = '0.8.2'.freeze
2
+ VERSION = '1.0.3'.freeze
3
3
  end
metadata CHANGED
@@ -1,14 +1,14 @@
1
1
  --- !ruby/object:Gem::Specification
2
2
  name: good_job
3
3
  version: !ruby/object:Gem::Version
4
- version: 0.8.2
4
+ version: 1.0.3
5
5
  platform: ruby
6
6
  authors:
7
7
  - Ben Sheldon
8
8
  autorequire:
9
9
  bindir: exe
10
10
  cert_chain: []
11
- date: 2020-07-18 00:00:00.000000000 Z
11
+ date: 2020-07-26 00:00:00.000000000 Z
12
12
  dependencies:
13
13
  - !ruby/object:Gem::Dependency
14
14
  name: concurrent-ruby
@@ -236,12 +236,15 @@ files:
236
236
  - README.md
237
237
  - exe/good_job
238
238
  - lib/active_job/queue_adapters/good_job_adapter.rb
239
+ - lib/generators/good_job/install_generator.rb
240
+ - lib/generators/good_job/templates/migration.rb.erb
239
241
  - lib/good_job.rb
240
242
  - lib/good_job/adapter.rb
241
243
  - lib/good_job/cli.rb
242
244
  - lib/good_job/job.rb
243
245
  - lib/good_job/lockable.rb
244
246
  - lib/good_job/logging.rb
247
+ - lib/good_job/performer.rb
245
248
  - lib/good_job/pg_locks.rb
246
249
  - lib/good_job/railtie.rb
247
250
  - lib/good_job/scheduler.rb