good_job 1.2.5 → 1.3.3

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: ca2673887424565881a47ad6eaf9300b6281e43c87066f01ceb899b76716d43e
4
- data.tar.gz: 6a0f749171316300ebfc7257d1d62efc330092391f026d6f50e20ccda5fee208
3
+ metadata.gz: d48414f6b03482087018157412c87afb3fdb70929a292294f7db2e14309c31fd
4
+ data.tar.gz: dd221c88385350fff14b593a6504bc98adeae44893d523c90949332a81eb6fd9
5
5
  SHA512:
6
- metadata.gz: 4fcef37d707e0d25f44965949abfeba914a4ee9f743ed32498e28df95751f666bdfb942c12049b23d78ff2b52e14465ca1cda2007bfae3541f9f0c53e6f74964
7
- data.tar.gz: db7d224567bde1427210638226d5005dd279a2c73c97ff9c19eb40edc523ae7863973cccf7d263a08dcf56ec66fd112a492afa269dad4b12cc70b79027baf296
6
+ metadata.gz: 77ab00b0cd0772641a402205856d5c03c675dc2edc769e5fed8909636fc2c4a69b033da98a7d0ae09a7ac1fc3f40dd5dc4ff637b3d13e4c290acb9a40bc51ead
7
+ data.tar.gz: 1777e663267f7e626d7f5ebf5b61b156e83b7bcf9234addddabeb10ef1054c05f8602ca0111c1abe4de9fb53b32aafac4df5d88e1c7c0147e0f1a7cbdc57219d
@@ -1,5 +1,86 @@
1
1
  # Changelog
2
2
 
3
+ ## [v1.3.3](https://github.com/bensheldon/good_job/tree/v1.3.3) (2020-12-01)
4
+
5
+ [Full Changelog](https://github.com/bensheldon/good_job/compare/v1.3.2...v1.3.3)
6
+
7
+ **Merged pull requests:**
8
+
9
+ - UI: Admin UI with filters and space efficient layout [\#173](https://github.com/bensheldon/good_job/pull/173) ([zealot128](https://github.com/zealot128))
10
+
11
+ ## [v1.3.2](https://github.com/bensheldon/good_job/tree/v1.3.2) (2020-11-12)
12
+
13
+ [Full Changelog](https://github.com/bensheldon/good_job/compare/v1.3.1...v1.3.2)
14
+
15
+ **Fixed bugs:**
16
+
17
+ - \(bug\) MultiScheduler polling bug [\#171](https://github.com/bensheldon/good_job/issues/171)
18
+ - MultiScheduler should delegate to all schedulers when state is nil [\#172](https://github.com/bensheldon/good_job/pull/172) ([bensheldon](https://github.com/bensheldon))
19
+
20
+ ## [v1.3.1](https://github.com/bensheldon/good_job/tree/v1.3.1) (2020-11-01)
21
+
22
+ [Full Changelog](https://github.com/bensheldon/good_job/compare/v1.3.0...v1.3.1)
23
+
24
+ **Implemented enhancements:**
25
+
26
+ - Extract polling from scheduler into Polling object [\#128](https://github.com/bensheldon/good_job/issues/128)
27
+ - Format serialized params to ease reading [\#170](https://github.com/bensheldon/good_job/pull/170) ([morgoth](https://github.com/morgoth))
28
+
29
+ **Fixed bugs:**
30
+
31
+ - Don't disconnect a nil activerecord connection [\#161](https://github.com/bensheldon/good_job/pull/161) ([bensheldon](https://github.com/bensheldon))
32
+
33
+ **Closed issues:**
34
+
35
+ - Propose addition of GoodJob to queue-shootout benchmarks [\#40](https://github.com/bensheldon/good_job/issues/40)
36
+
37
+ **Merged pull requests:**
38
+
39
+ - Ensure Rails is a development dependency [\#169](https://github.com/bensheldon/good_job/pull/169) ([bensheldon](https://github.com/bensheldon))
40
+ - Fix Ruby 2.7 GH action by setting default bundler explicitly [\#166](https://github.com/bensheldon/good_job/pull/166) ([bensheldon](https://github.com/bensheldon))
41
+ - Cache ruby version explicitly in Github Action [\#165](https://github.com/bensheldon/good_job/pull/165) ([bensheldon](https://github.com/bensheldon))
42
+ - Update development dependencies, rubocop [\#164](https://github.com/bensheldon/good_job/pull/164) ([bensheldon](https://github.com/bensheldon))
43
+ - Fix intended constant hierarchy of GoodJob::Scheduler::ThreadPoolExecutor [\#158](https://github.com/bensheldon/good_job/pull/158) ([bensheldon](https://github.com/bensheldon))
44
+ - Add bin/test\_app executable for Rails debugging [\#157](https://github.com/bensheldon/good_job/pull/157) ([bensheldon](https://github.com/bensheldon))
45
+ - Extract Scheduler polling behavior to its own object [\#152](https://github.com/bensheldon/good_job/pull/152) ([bensheldon](https://github.com/bensheldon))
46
+
47
+ ## [v1.3.0](https://github.com/bensheldon/good_job/tree/v1.3.0) (2020-10-03)
48
+
49
+ [Full Changelog](https://github.com/bensheldon/good_job/compare/v1.2.6...v1.3.0)
50
+
51
+ **Implemented enhancements:**
52
+
53
+ - Lengthen default poll interval from 1 to 5 seconds [\#156](https://github.com/bensheldon/good_job/pull/156) ([bensheldon](https://github.com/bensheldon))
54
+
55
+ **Merged pull requests:**
56
+
57
+ - Rename reperform\_jobs\_on\_standard\_error to retry\_on\_unhandled\_error [\#154](https://github.com/bensheldon/good_job/pull/154) ([morgoth](https://github.com/morgoth))
58
+
59
+ ## [v1.2.6](https://github.com/bensheldon/good_job/tree/v1.2.6) (2020-09-29)
60
+
61
+ [Full Changelog](https://github.com/bensheldon/good_job/compare/v1.2.5...v1.2.6)
62
+
63
+ **Implemented enhancements:**
64
+
65
+ - Preserve only failed jobs [\#136](https://github.com/bensheldon/good_job/issues/136)
66
+ - Add `GoodJob.preserve\_job\_records = :on\_unhandled\_error` option to only preserve jobs that errored [\#145](https://github.com/bensheldon/good_job/pull/145) ([morgoth](https://github.com/morgoth))
67
+
68
+ **Fixed bugs:**
69
+
70
+ - Fix LogSubscriber notifications for finished\_timer\_task and finished\_job\_task [\#148](https://github.com/bensheldon/good_job/pull/148) ([bensheldon](https://github.com/bensheldon))
71
+
72
+ **Closed issues:**
73
+
74
+ - run-once guarantee? [\#151](https://github.com/bensheldon/good_job/issues/151)
75
+
76
+ **Merged pull requests:**
77
+
78
+ - Add info how to setup basic auth for engine [\#153](https://github.com/bensheldon/good_job/pull/153) ([morgoth](https://github.com/morgoth))
79
+ - Add documentation for Dashboard Rails::Engine [\#149](https://github.com/bensheldon/good_job/pull/149) ([bensheldon](https://github.com/bensheldon))
80
+ - Style cleanup to Job error handling [\#147](https://github.com/bensheldon/good_job/pull/147) ([bensheldon](https://github.com/bensheldon))
81
+ - Replace gerund titles in Readme [\#146](https://github.com/bensheldon/good_job/pull/146) ([bensheldon](https://github.com/bensheldon))
82
+ - Only allow Scheduler to be initialized with max\_threads and poll\_interval; remove full access to pool and timer\_task options [\#137](https://github.com/bensheldon/good_job/pull/137) ([bensheldon](https://github.com/bensheldon))
83
+
3
84
  ## [v1.2.5](https://github.com/bensheldon/good_job/tree/v1.2.5) (2020-09-17)
4
85
 
5
86
  [Full Changelog](https://github.com/bensheldon/good_job/compare/v1.2.4...v1.2.5)
@@ -7,10 +88,15 @@
7
88
  **Implemented enhancements:**
8
89
 
9
90
  - Use Zeitwerk for auto-loading [\#87](https://github.com/bensheldon/good_job/issues/87)
91
+ - Spike on data dashboard; pull in full Bootstrap CSS and JS [\#131](https://github.com/bensheldon/good_job/pull/131) ([bensheldon](https://github.com/bensheldon))
10
92
 
11
93
  **Fixed bugs:**
12
94
 
13
95
  - `poll-interval=-1` does not disable polling as intended [\#133](https://github.com/bensheldon/good_job/issues/133)
96
+ - Update Gemspec to reflect that GoodJob is not compatible with Rails 5.1 [\#143](https://github.com/bensheldon/good_job/pull/143) ([bensheldon](https://github.com/bensheldon))
97
+ - Prevent jobs hanging [\#141](https://github.com/bensheldon/good_job/pull/141) ([morgoth](https://github.com/morgoth))
98
+ - Add explicit require\_paths to gemspec for engine [\#134](https://github.com/bensheldon/good_job/pull/134) ([bensheldon](https://github.com/bensheldon))
99
+ - Use `connection.quote\_table\_name` and add spacing for SQL concatenation [\#124](https://github.com/bensheldon/good_job/pull/124) ([bensheldon](https://github.com/bensheldon))
14
100
 
15
101
  **Closed issues:**
16
102
 
@@ -21,17 +107,12 @@
21
107
  **Merged pull requests:**
22
108
 
23
109
  - Test GoodJob against Rails HEAD [\#144](https://github.com/bensheldon/good_job/pull/144) ([bensheldon](https://github.com/bensheldon))
24
- - Update Gemspec to reflect that GoodJob is not compatible with Rails 5.1 [\#143](https://github.com/bensheldon/good_job/pull/143) ([bensheldon](https://github.com/bensheldon))
25
110
  - Drop Ruby 2.4 support [\#142](https://github.com/bensheldon/good_job/pull/142) ([morgoth](https://github.com/morgoth))
26
- - Prevent jobs hanging [\#141](https://github.com/bensheldon/good_job/pull/141) ([morgoth](https://github.com/morgoth))
27
111
  - Remove arguments from perform method [\#140](https://github.com/bensheldon/good_job/pull/140) ([morgoth](https://github.com/morgoth))
28
112
  - Extract "execute" method to reduce "perform" method complexity [\#138](https://github.com/bensheldon/good_job/pull/138) ([morgoth](https://github.com/morgoth))
29
113
  - Correct example on how to configure multiple queues by command line. [\#135](https://github.com/bensheldon/good_job/pull/135) ([morgoth](https://github.com/morgoth))
30
- - Add explicit require\_paths to gemspec for engine [\#134](https://github.com/bensheldon/good_job/pull/134) ([bensheldon](https://github.com/bensheldon))
31
- - Spike on data dashboard; pull in full Bootstrap CSS and JS [\#131](https://github.com/bensheldon/good_job/pull/131) ([bensheldon](https://github.com/bensheldon))
32
114
  - Update ActionMailer Job class, to match the default [\#130](https://github.com/bensheldon/good_job/pull/130) ([morgoth](https://github.com/morgoth))
33
115
  - Add initial Engine scaffold [\#125](https://github.com/bensheldon/good_job/pull/125) ([bensheldon](https://github.com/bensheldon))
34
- - Use `connection.quote\_table\_name` and add spacing for SQL concatenation [\#124](https://github.com/bensheldon/good_job/pull/124) ([bensheldon](https://github.com/bensheldon))
35
116
  - Zeitwerk Loader Implementation [\#123](https://github.com/bensheldon/good_job/pull/123) ([gadimbaylisahil](https://github.com/gadimbaylisahil))
36
117
  - Update code-level documentation [\#111](https://github.com/bensheldon/good_job/pull/111) ([bensheldon](https://github.com/bensheldon))
37
118
 
@@ -42,6 +123,11 @@
42
123
  **Implemented enhancements:**
43
124
 
44
125
  - Add environment variable to mirror `cleanup\_preserved\_jobs --before-seconds-ago=SECONDS` [\#110](https://github.com/bensheldon/good_job/issues/110)
126
+ - Allow env variable config for cleanups [\#114](https://github.com/bensheldon/good_job/pull/114) ([gadimbaylisahil](https://github.com/gadimbaylisahil))
127
+
128
+ **Fixed bugs:**
129
+
130
+ - Better table name detection for Job queries [\#119](https://github.com/bensheldon/good_job/pull/119) ([gadimbaylisahil](https://github.com/gadimbaylisahil))
45
131
 
46
132
  **Closed issues:**
47
133
 
@@ -52,9 +138,7 @@
52
138
  **Merged pull requests:**
53
139
 
54
140
  - Remove unused PgLocks class [\#120](https://github.com/bensheldon/good_job/pull/120) ([gadimbaylisahil](https://github.com/gadimbaylisahil))
55
- - Better table name detection for Job queries [\#119](https://github.com/bensheldon/good_job/pull/119) ([gadimbaylisahil](https://github.com/gadimbaylisahil))
56
141
  - Fix readme CommandLine option links [\#115](https://github.com/bensheldon/good_job/pull/115) ([gadimbaylisahil](https://github.com/gadimbaylisahil))
57
- - Allow env variable config for cleanups [\#114](https://github.com/bensheldon/good_job/pull/114) ([gadimbaylisahil](https://github.com/gadimbaylisahil))
58
142
  - Have YARD render markdown files with GFM \(Github Flavored Markdown\) [\#113](https://github.com/bensheldon/good_job/pull/113) ([bensheldon](https://github.com/bensheldon))
59
143
  - Add markdownlint to lint readme [\#109](https://github.com/bensheldon/good_job/pull/109) ([bensheldon](https://github.com/bensheldon))
60
144
  - Remove unused method in PgLocks [\#107](https://github.com/bensheldon/good_job/pull/107) ([gadimbaylisahil](https://github.com/gadimbaylisahil))
@@ -71,6 +155,7 @@
71
155
  **Merged pull requests:**
72
156
 
73
157
  - stop depending on all rails libs [\#104](https://github.com/bensheldon/good_job/pull/104) ([thilo](https://github.com/thilo))
158
+ - Use more ActiveRecord in Lockable and not connection.execute [\#102](https://github.com/bensheldon/good_job/pull/102) ([bensheldon](https://github.com/bensheldon))
74
159
 
75
160
  ## [v1.2.2](https://github.com/bensheldon/good_job/tree/v1.2.2) (2020-08-27)
76
161
 
@@ -91,7 +176,6 @@
91
176
 
92
177
  **Merged pull requests:**
93
178
 
94
- - Use more ActiveRecord in Lockable and not connection.execute [\#102](https://github.com/bensheldon/good_job/pull/102) ([bensheldon](https://github.com/bensheldon))
95
179
  - Run CI tests on Ruby 2.5, 2.6, and 2.7 [\#101](https://github.com/bensheldon/good_job/pull/101) ([arku](https://github.com/arku))
96
180
  - Return to using executor.wrap around Scheduler execution task [\#99](https://github.com/bensheldon/good_job/pull/99) ([bensheldon](https://github.com/bensheldon))
97
181
  - Fix Ruby 2.7 keyword arguments warning [\#98](https://github.com/bensheldon/good_job/pull/98) ([arku](https://github.com/arku))
@@ -354,7 +438,6 @@
354
438
  - Add pg gem as explicit dependency [\#13](https://github.com/bensheldon/good_job/pull/13) ([bensheldon](https://github.com/bensheldon))
355
439
  - Bump nokogiri from 1.10.7 to 1.10.9 [\#12](https://github.com/bensheldon/good_job/pull/12) ([dependabot[bot]](https://github.com/apps/dependabot))
356
440
  - Add Appraisal with tests for Rails 5.1, 5.2, 6.0 [\#11](https://github.com/bensheldon/good_job/pull/11) ([bensheldon](https://github.com/bensheldon))
357
- - Use Rails.logger and ActiveSupport::Notifications for logging instead of puts [\#10](https://github.com/bensheldon/good_job/pull/10) ([bensheldon](https://github.com/bensheldon))
358
441
 
359
442
  ## [v0.2.0](https://github.com/bensheldon/good_job/tree/v0.2.0) (2020-03-06)
360
443
 
@@ -362,6 +445,7 @@
362
445
 
363
446
  **Merged pull requests:**
364
447
 
448
+ - Use Rails.logger and ActiveSupport::Notifications for logging instead of puts [\#10](https://github.com/bensheldon/good_job/pull/10) ([bensheldon](https://github.com/bensheldon))
365
449
  - Remove minitest files [\#9](https://github.com/bensheldon/good_job/pull/9) ([bensheldon](https://github.com/bensheldon))
366
450
  - Use scheduled\_at and priority for scheduling [\#8](https://github.com/bensheldon/good_job/pull/8) ([bensheldon](https://github.com/bensheldon))
367
451
  - Create Github Action workflow for PRs and Issues [\#7](https://github.com/bensheldon/good_job/pull/7) ([bensheldon](https://github.com/bensheldon))
data/README.md CHANGED
@@ -1,5 +1,8 @@
1
1
  # GoodJob
2
2
 
3
+ [![Gem Version](https://badge.fury.io/rb/good_job.svg)](https://rubygems.org/gems/good_job)
4
+ [![Test Status](https://github.com/bensheldon/good_job/workflows/Test/badge.svg)](https://github.com/bensheldon/good_job/actions)
5
+
3
6
  GoodJob is a multithreaded, Postgres-based, ActiveJob backend for Ruby on Rails.
4
7
 
5
8
  **Inspired by [Delayed::Job](https://github.com/collectiveidea/delayed_job) and [Que](https://github.com/que-rb/que), GoodJob is designed for maximum compatibility with Ruby on Rails, ActiveJob, and Postgres to be simple and performant for most workloads.**
@@ -33,7 +36,8 @@ For more of the story of GoodJob, read the [introductory blog post](https://isla
33
36
  - [`good_job cleanup_preserved_jobs`](#good_job-cleanup_preserved_jobs)
34
37
  - [Adapter options](#adapter-options)
35
38
  - [Global options](#global-options)
36
- - [Going deeper](#going-deeper)
39
+ - [Dashboard](#dashboard)
40
+ - [Go deeper](#go-deeper)
37
41
  - [Exceptions, retries, and reliability](#exceptions-retries-and-reliability)
38
42
  - [Exceptions](#exceptions)
39
43
  - [Retries](#retries)
@@ -41,12 +45,12 @@ For more of the story of GoodJob, read the [introductory blog post](https://isla
41
45
  - [Timeouts](#timeouts)
42
46
  - [Optimize queues, threads, and processes](#optimize-queues-threads-and-processes)
43
47
  - [Database connections](#database-connections)
44
- - [Executing jobs async / in-process](#executing-jobs-async--in-process)
45
- - [Migrating to GoodJob from a different ActiveJob backend](#migrating-to-goodjob-from-a-different-activejob-backend)
46
- - [Monitoring and preserving worked jobs](#monitoring-and-preserving-worked-jobs)
47
- - [Contributing](#contributing)
48
+ - [Execute jobs async / in-process](#execute-jobs-async--in-process)
49
+ - [Migrate to GoodJob from a different ActiveJob backend](#migrate-to-goodjob-from-a-different-activejob-backend)
50
+ - [Monitor and preserve worked jobs](#monitor-and-preserve-worked-jobs)
51
+ - [Contribute](#contribute)
48
52
  - [Gem development](#gem-development)
49
- - [Releasing](#releasing)
53
+ - [Release](#release)
50
54
  - [License](#license)
51
55
 
52
56
  ## Set up
@@ -209,7 +213,7 @@ Good Job’s general behavior can also be configured via several attributes dire
209
213
 
210
214
  - **`GoodJob.logger`** ([Rails Logger](https://api.rubyonrails.org/classes/ActiveSupport/Logger.html)) lets you set a custom logger for GoodJob. It should be an instance of a Rails `Logger`.
211
215
  - **`GoodJob.preserve_job_records`** (boolean) keeps job records in your database even after jobs are completed. (Default: `false`)
212
- - **`GoodJob.reperform_jobs_on_standard_error`** (boolean) causes jobs to be re-queued and retried if they raise an instance of `StandardError`. Instances of `Exception`, like SIGINT, will *always* be retried, regardless of this attribute’s value. (Default: `true`)
216
+ - **`GoodJob.retry_on_unhandled_error`** (boolean) causes jobs to be re-queued and retried if they raise an instance of `StandardError`. Instances of `Exception`, like SIGINT, will *always* be retried, regardless of this attribute’s value. (Default: `true`)
213
217
  - **`GoodJob.on_thread_error`** (proc, lambda, or callable) will be called when an Exception. It can be useful for logging errors to bug tracking services, like Sentry or Airbrake.
214
218
 
215
219
  You’ll generally want to configure these in `config/initializers/good_job.rb`, like so:
@@ -217,11 +221,56 @@ You’ll generally want to configure these in `config/initializers/good_job.rb`,
217
221
  ```ruby
218
222
  # config/initializers/good_job.rb
219
223
  GoodJob.preserve_job_records = true
220
- GoodJob.reperform_jobs_on_standard_error = false
224
+ GoodJob.retry_on_unhandled_error = false
221
225
  GoodJob.on_thread_error = -> (exception) { Raven.capture_exception(exception) }
222
226
  ```
223
227
 
224
- ## Going deeper
228
+ ### Dashboard
229
+
230
+ _🚧 GoodJob's dashboard is a work in progress. Please contribute ideas and code on [Github](https://github.com/bensheldon/good_job/issues)._
231
+
232
+ GoodJob includes a Dashboard as a mountable `Rails::Engine`.
233
+
234
+ 1. Explicitly require the Engine code at the top of your `config/application.rb` file, immediately after Rails is required. This is necessary because the mountable engine is an optional feature of GoodJob.
235
+
236
+ ```ruby
237
+ # config/application.rb
238
+ require_relative 'boot'
239
+
240
+ require 'rails/all'
241
+ require 'good_job/engine' # <= Add this line
242
+ # ...
243
+ ```
244
+
245
+ 1. Mount the engine in your `config/routes.rb` file. The following will mount it at `http://example.com/good_job`.
246
+
247
+ ```ruby
248
+ # config/routes.rb
249
+ # ...
250
+ mount GoodJob::Engine => 'good_job'
251
+ ```
252
+
253
+ Because jobs can potentially contain sensitive information, you should authorize access. For example, using Devise's `authenticate` helper, that might look like:
254
+
255
+ ```ruby
256
+ # config/routes.rb
257
+ # ...
258
+ authenticate :user, ->(user) { user.admin? } do
259
+ mount GoodJob::Engine => 'good_job'
260
+ end
261
+ ```
262
+
263
+ Another option is using basic auth like this:
264
+
265
+ ```ruby
266
+ # config/initializers/good_job.rb
267
+ GoodJob::Engine.middleware.use(Rack::Auth::Basic) do |username, password|
268
+ ActiveSupport::SecurityUtils.secure_compare(Rails.application.credentials.good_job_username, username) &&
269
+ ActiveSupport::SecurityUtils.secure_compare(Rails.application.credentials.good_job_password, password)
270
+ end
271
+ ```
272
+
273
+ ## Go deeper
225
274
 
226
275
  ### Exceptions, retries, and reliability
227
276
 
@@ -255,7 +304,7 @@ When using `retry_on` with _a limited number of retries_, the final exception wi
255
304
 
256
305
  ```ruby
257
306
  # config/initializers/good_job.rb
258
- GoodJob.reperform_jobs_on_standard_error = false
307
+ GoodJob.retry_on_unhandled_error = false
259
308
  ```
260
309
 
261
310
  Alternatively, pass a block to `retry_on` to handle the final exception instead of raising it to GoodJob:
@@ -380,7 +429,7 @@ Each GoodJob execution thread requires its own database connection that is autom
380
429
  pool: <%= [ENV.fetch("RAILS_MAX_THREADS", 5).to_i, ENV.fetch("GOOD_JOB_MAX_THREADS", 4).to_i].max %>
381
430
  ```
382
431
 
383
- ### Executing jobs async / in-process
432
+ ### Execute jobs async / in-process
384
433
 
385
434
  GoodJob can execute jobs "async" in the same process as the webserver (e.g. `bin/rail s`). GoodJob's async execution mode offers benefits of economy by not requiring a separate job worker process, but with the tradeoff of increased complexity. Async mode can be configured in two ways:
386
435
 
@@ -431,7 +480,7 @@ Depending on your application configuration, you may need to take additional ste
431
480
 
432
481
  GoodJob is compatible with Puma's `preload_app!` method.
433
482
 
434
- ### Migrating to GoodJob from a different ActiveJob backend
483
+ ### Migrate to GoodJob from a different ActiveJob backend
435
484
 
436
485
  If your application is already using an ActiveJob backend, you will need to install GoodJob to enqueue and perform newly created jobs _and_ finish performing pre-existing jobs on the previous backend.
437
486
 
@@ -455,7 +504,7 @@ If your application is already using an ActiveJob backend, you will need to inst
455
504
 
456
505
  1. Once you are confident that no unperformed jobs remain in the previous ActiveJob backend, code and configuration for that backend can be completely removed.
457
506
 
458
- ### Monitoring and preserving worked jobs
507
+ ### Monitor and preserve worked jobs
459
508
 
460
509
  GoodJob is fully instrumented with [`ActiveSupport::Notifications`](https://edgeguides.rubyonrails.org/active_support_instrumentation.html#introduction-to-instrumentation).
461
510
 
@@ -482,7 +531,7 @@ It is also necessary to delete these preserved jobs from the database after a ce
482
531
  $ bundle exec good_job cleanup_preserved_jobs --before-seconds-ago=86400
483
532
  ```
484
533
 
485
- ## Contributing
534
+ ## Contribute
486
535
 
487
536
  Contributions are welcomed and appreciated 🙏
488
537
 
@@ -527,7 +576,7 @@ $ bundle install
527
576
  # => Using good_job 0.1.0 from https://github.com/bensheldon/good_job.git (at /Users/You/Projects/good_job@dc57fb0)
528
577
  ```
529
578
 
530
- ### Releasing
579
+ ### Release
531
580
 
532
581
  Package maintainers can release this gem by running:
533
582
 
@@ -1,10 +1,59 @@
1
1
  module GoodJob
2
2
  class DashboardsController < GoodJob::BaseController
3
- def index
4
- @jobs = GoodJob::Job.display_all(after_scheduled_at: params[:after_scheduled_at], after_id: params[:after_id])
3
+ class JobFilter
4
+ attr_accessor :params
5
+
6
+ def initialize(params)
7
+ @params = params
8
+ end
9
+
10
+ def last
11
+ @_last ||= jobs.last
12
+ end
13
+
14
+ def jobs
15
+ sql = GoodJob::Job.display_all(after_scheduled_at: params[:after_scheduled_at], after_id: params[:after_id])
5
16
  .limit(params.fetch(:limit, 10))
17
+ if params[:job_class] # rubocop:disable Style/IfUnlessModifier
18
+ sql = sql.where("serialized_params->>'job_class' = ?", params[:job_class])
19
+ end
20
+ if params[:state]
21
+ case params[:state]
22
+ when 'finished'
23
+ sql = sql.finished
24
+ when 'unfinished'
25
+ sql = sql.unfinished
26
+ when 'errors'
27
+ sql = sql.where.not(error: nil)
28
+ end
29
+ end
30
+ sql
31
+ end
32
+
33
+ def states
34
+ {
35
+ 'finished' => GoodJob::Job.finished.count,
36
+ 'unfinished' => GoodJob::Job.unfinished.count,
37
+ 'errors' => GoodJob::Job.where.not(error: nil).count,
38
+ }
39
+ end
40
+
41
+ def job_classes
42
+ GoodJob::Job.group("serialized_params->>'job_class'").count
43
+ end
44
+
45
+ def to_query(override)
46
+ {
47
+ state: params[:state],
48
+ job_class: params[:job_class],
49
+ }.merge(override).delete_if { |_, v| v.nil? }.to_query
50
+ end
51
+ end
52
+
53
+ def index
54
+ @filter = JobFilter.new(params)
6
55
 
7
- job_data = GoodJob::Job.connection.exec_query Arel.sql(<<~SQL)
56
+ job_data = GoodJob::Job.connection.exec_query Arel.sql(<<~SQL.squish)
8
57
  SELECT *
9
58
  FROM generate_series(
10
59
  date_trunc('hour', NOW() - '1 day'::interval),
@@ -2,13 +2,36 @@
2
2
  <%= render 'shared/chart', chart_data: @chart %>
3
3
  </div>
4
4
 
5
- <% if @jobs.present? %>
6
- <%= render 'shared/jobs_table', jobs: @jobs %>
5
+ <div class='card mb-2'>
6
+ <div class='card-body d-flex flex-wrap'>
7
+ <div class='mr-4'>
8
+ <small>Filter by job class</small>
9
+ <br>
10
+ <% @filter.job_classes.each do |name, count| %>
11
+ <a href='<%= request.path + "?#{@filter.to_query(job_class: name)}" %>' class='btn btn-sm btn-outline-secondary <%= "active" if params[:job_class] == name %>'>
12
+ <%= name %> (<%= count %>)
13
+ </a>
14
+ <% end %>
15
+ </div>
16
+ <div>
17
+ <small>Filter by state</small>
18
+ <br>
19
+ <% @filter.states.each do |name, count| %>
20
+ <a href='<%= request.path + "?#{@filter.to_query(state: name)}" %>' class='btn btn-sm btn-outline-secondary <%= "active" if params[:state] == name %>'>
21
+ <%= name %> (<%= count %>)
22
+ </a>
23
+ <% end %>
24
+ </div>
25
+ </div>
26
+ </div>
27
+
28
+ <% if @filter.jobs.present? %>
29
+ <%= render 'shared/jobs_table', jobs: @filter.jobs %>
7
30
 
8
31
  <nav aria-label="Job pagination">
9
32
  <ul class="pagination">
10
33
  <li class="page-item">
11
- <%= link_to({ after_scheduled_at: (@jobs.last.scheduled_at || @jobs.last.created_at), after_id: @jobs.last.id }, class: "page-link") do %>
34
+ <%= link_to({ after_scheduled_at: (@filter.last.scheduled_at || @filter.last.created_at), after_id: @filter.last.id }, class: "page-link") do %>
12
35
  Next jobs <span aria-hidden="true">&raquo;</span>
13
36
  <% end %>
14
37
  </li>
@@ -18,7 +18,7 @@
18
18
  </head>
19
19
  <body>
20
20
  <nav class="navbar navbar-expand-lg navbar-light bg-light">
21
- <div class="container">
21
+ <div class="container-fluid">
22
22
  <%= link_to "GoodJob 👍", root_path, class: 'navbar-brand mb-0 h1' %>
23
23
  <button class="navbar-toggler" type="button" data-toggle="collapse" data-target="#navbarSupportedContent" aria-controls="navbarSupportedContent" aria-expanded="false" aria-label="Toggle navigation">
24
24
  <span class="navbar-toggler-icon"></span>
@@ -27,23 +27,34 @@
27
27
  <div class="collapse navbar-collapse" id="navbarSupportedContent">
28
28
  <ul class="navbar-nav mr-auto">
29
29
  <li class="nav-item">
30
- <%= link_to "All jobs", root_path, class: ["nav-link", ("active" if current_page?(root_path))] %>
31
- </li>
32
- <li class="nav-item">
33
- <%= link_to "Upcoming Jobs", 'todo', class: ["nav-link", ("active" if current_page?('todo'))] %>
34
- </li>
35
- <li class="nav-item">
36
- <%= link_to "Finished Jobs", 'todo', class: ["nav-link", ("active" if current_page?('todo'))] %>
37
- </li>
38
- <li class="nav-item">
39
- <%= link_to "Errored Jobs", 'todo', class: ["nav-link", ("active" if current_page?('todo'))] %>
30
+ <%= link_to root_path, class: ["nav-link", ("active" if current_page?(root_path))] do %>
31
+ All jobs <span class="badge badge-secondary">More views coming soon</span>
32
+ <% end %>
40
33
  </li>
34
+
35
+ <!-- Coming Soon
36
+ <li class="nav-item">
37
+ <%= link_to "Upcoming Jobs", 'todo', class: ["nav-link", ("active" if current_page?('todo'))] %>
38
+ </li>
39
+ <li class="nav-item">
40
+ <%= link_to "Finished Jobs", 'todo', class: ["nav-link", ("active" if current_page?('todo'))] %>
41
+ </li>
42
+ <li class="nav-item">
43
+ <%= link_to "Errored Jobs", 'todo', class: ["nav-link", ("active" if current_page?('todo'))] %>
44
+ </li>
45
+ -->
41
46
  </ul>
42
47
  </div>
43
48
  </div>
44
49
  </nav>
45
50
 
46
- <div class="container">
51
+ <div class="container-fluid">
52
+ <div class="card border-warning text-dark my-3">
53
+ <div class="card-body">
54
+ <p class="card-text">🚧 GoodJob's dashboard is a work in progress. Please contribute ideas and code on <a href="https://github.com/bensheldon/good_job/issues" target="_blank" rel="nofollow noopener noreferrer">Github</a>.</p>
55
+ </div>
56
+ </div>
57
+
47
58
  <%= yield %>
48
59
  </div>
49
60
  </body>
@@ -1,13 +1,13 @@
1
1
  <div class="table-responsive">
2
- <table class="table table-bordered table-hover">
2
+ <table class="table table-bordered table-hover table-sm">
3
3
  <thead>
4
4
  <th>GoodJob ID</th>
5
5
  <th>ActiveJob ID</th>
6
6
  <th>Job Class</th>
7
7
  <th>Queue</th>
8
8
  <th>Scheduled At</th>
9
- <th>ActiveJob Params</th>
10
9
  <th>Error</th>
10
+ <th>ActiveJob Params</th>
11
11
  </thead>
12
12
  <tbody>
13
13
  <% jobs.each do |job| %>
@@ -17,8 +17,8 @@
17
17
  <td><%= job.serialized_params['job_class'] %></td>
18
18
  <td><%= job.queue_name %></td>
19
19
  <td><%= job.scheduled_at || job.created_at %></td>
20
- <td><%= job.serialized_params %></td>
21
20
  <td><%= job.error %></td>
21
+ <td><pre><%= JSON.pretty_generate(job.serialized_params) %></pre></td>
22
22
  </tr>
23
23
  <% end %>
24
24
  </tbody>
@@ -32,18 +32,35 @@ module GoodJob
32
32
  # Whether to preserve job records in the database after they have finished (default: +false+).
33
33
  # By default, GoodJob deletes job records after the job is completed successfully.
34
34
  # If you want to preserve jobs for latter inspection, set this to +true+.
35
+ # If you want to preserve only jobs that finished with error for latter inspection, set this to +:on_unhandled_error+.
35
36
  # If +true+, you will need to clean out jobs using the +good_job cleanup_preserved_jobs+ CLI command.
36
37
  # @return [Boolean]
37
38
  mattr_accessor :preserve_job_records, default: false
38
39
 
39
- # @!attribute [rw] reperform_jobs_on_standard_error
40
+ # @!attribute [rw] retry_on_unhandled_error
40
41
  # @!scope class
41
42
  # Whether to re-perform a job when a type of +StandardError+ is raised to GoodJob (default: +true+).
42
43
  # If +true+, causes jobs to be re-queued and retried if they raise an instance of +StandardError+.
43
44
  # If +false+, jobs will be discarded or marked as finished if they raise an instance of +StandardError+.
44
45
  # Instances of +Exception+, like +SIGINT+, will *always* be retried, regardless of this attribute's value.
45
46
  # @return [Boolean]
46
- mattr_accessor :reperform_jobs_on_standard_error, default: true
47
+ mattr_accessor :retry_on_unhandled_error, default: true
48
+
49
+ # @deprecated Use {GoodJob#retry_on_unhandled_error} instead.
50
+ def self.reperform_jobs_on_standard_error
51
+ ActiveSupport::Deprecation.warn(
52
+ "Calling 'GoodJob.reperform_jobs_on_standard_error' is deprecated. Please use 'retry_on_unhandled_error'"
53
+ )
54
+ retry_on_unhandled_error
55
+ end
56
+
57
+ # @deprecated Use {GoodJob#retry_on_unhandled_error=} instead.
58
+ def self.reperform_jobs_on_standard_error=(value)
59
+ ActiveSupport::Deprecation.warn(
60
+ "Setting 'GoodJob.reperform_jobs_on_standard_error=' is deprecated. Please use 'retry_on_unhandled_error='"
61
+ )
62
+ self.retry_on_unhandled_error = value
63
+ end
47
64
 
48
65
  # @!attribute [rw] on_thread_error
49
66
  # @!scope class
@@ -43,8 +43,10 @@ module GoodJob
43
43
 
44
44
  if @execution_mode == :async # rubocop:disable Style/GuardClause
45
45
  @notifier = notifier || GoodJob::Notifier.new
46
+ @poller = GoodJob::Poller.new(poll_interval: configuration.poll_interval)
46
47
  @scheduler = scheduler || GoodJob::Scheduler.from_configuration(configuration)
47
48
  @notifier.recipients << [@scheduler, :create_thread]
49
+ @poller.recipients << [@scheduler, :create_thread]
48
50
  end
49
51
  end
50
52
 
@@ -88,6 +90,7 @@ module GoodJob
88
90
  # @return [void]
89
91
  def shutdown(wait: true)
90
92
  @notifier&.shutdown(wait: wait)
93
+ @poller&.shutdown(wait: wait)
91
94
  @scheduler&.shutdown(wait: wait)
92
95
  end
93
96
 
@@ -42,14 +42,16 @@ module GoodJob
42
42
  method_option :poll_interval,
43
43
  type: :numeric,
44
44
  banner: 'SECONDS',
45
- desc: "Interval between polls for available jobs in seconds (env var: GOOD_JOB_POLL_INTERVAL, default: 1)"
45
+ desc: "Interval between polls for available jobs in seconds (env var: GOOD_JOB_POLL_INTERVAL, default: 5)"
46
46
  def start
47
47
  set_up_application!
48
+ configuration = GoodJob::Configuration.new(options)
48
49
 
49
50
  notifier = GoodJob::Notifier.new
50
- configuration = GoodJob::Configuration.new(options)
51
+ poller = GoodJob::Poller.new(poll_interval: configuration.poll_interval)
51
52
  scheduler = GoodJob::Scheduler.from_configuration(configuration)
52
53
  notifier.recipients << [scheduler, :create_thread]
54
+ poller.recipients << [scheduler, :create_thread]
53
55
 
54
56
  @stop_good_job_executable = false
55
57
  %w[INT TERM].each do |signal|
@@ -62,6 +64,7 @@ module GoodJob
62
64
  end
63
65
 
64
66
  notifier.shutdown
67
+ poller.shutdown
65
68
  scheduler.shutdown
66
69
  end
67
70
 
@@ -5,6 +5,13 @@ module GoodJob
5
5
  # set options to get the final values for each option.
6
6
  #
7
7
  class Configuration
8
+ # Default number of threads to use per {Scheduler}
9
+ DEFAULT_MAX_THREADS = 5
10
+ # Default number of seconds between polls for jobs
11
+ DEFAULT_POLL_INTERVAL = 5
12
+ # Default number of seconds to preserve jobs for {CLI#cleanup_preserved_jobs}
13
+ DEFAULT_CLEANUP_PRESERVED_JOBS_BEFORE_SECONDS_AGO = 24 * 60 * 60
14
+
8
15
  # @!attribute [r] options
9
16
  # The options that were explicitly set when initializing +Configuration+.
10
17
  # @return [Hash]
@@ -69,7 +76,7 @@ module GoodJob
69
76
  options[:max_threads] ||
70
77
  env['GOOD_JOB_MAX_THREADS'] ||
71
78
  env['RAILS_MAX_THREADS'] ||
72
- ActiveRecord::Base.connection_pool.size
79
+ DEFAULT_MAX_THREADS
73
80
  ).to_i
74
81
  end
75
82
 
@@ -92,7 +99,7 @@ module GoodJob
92
99
  (
93
100
  options[:poll_interval] ||
94
101
  env['GOOD_JOB_POLL_INTERVAL'] ||
95
- 1
102
+ DEFAULT_POLL_INTERVAL
96
103
  ).to_i
97
104
  end
98
105
 
@@ -100,7 +107,7 @@ module GoodJob
100
107
  (
101
108
  options[:before_seconds_ago] ||
102
109
  env['GOOD_JOB_CLEANUP_PRESERVED_JOBS_BEFORE_SECONDS_AGO'] ||
103
- 24 * 60 * 60
110
+ DEFAULT_CLEANUP_PRESERVED_JOBS_BEFORE_SECONDS_AGO
104
111
  ).to_i
105
112
  end
106
113
  end
@@ -189,37 +189,31 @@ module GoodJob
189
189
  self.performed_at = Time.current
190
190
  save! if GoodJob.preserve_job_records
191
191
 
192
- result, rescued_error = execute
192
+ result, unhandled_error = execute
193
193
 
194
- retry_or_discard_error = GoodJob::CurrentExecution.error_on_retry ||
195
- GoodJob::CurrentExecution.error_on_discard
196
-
197
- error = nil
198
- if rescued_error
199
- error = rescued_error
200
- elsif result.is_a?(Exception)
201
- error = result
194
+ result_error = nil
195
+ if result.is_a?(Exception)
196
+ result_error = result
202
197
  result = nil
203
- elsif retry_or_discard_error
204
- error = retry_or_discard_error
205
198
  end
206
199
 
207
- error_message = "#{error.class}: #{error.message}" if error
208
- self.error = error_message
200
+ job_error = unhandled_error ||
201
+ result_error ||
202
+ GoodJob::CurrentExecution.error_on_retry ||
203
+ GoodJob::CurrentExecution.error_on_discard
209
204
 
210
- if rescued_error && GoodJob.reperform_jobs_on_standard_error
205
+ self.error = "#{job_error.class}: #{job_error.message}" if job_error
206
+
207
+ if unhandled_error && GoodJob.retry_on_unhandled_error
211
208
  save!
212
- else
209
+ elsif GoodJob.preserve_job_records == true || (unhandled_error && GoodJob.preserve_job_records == :on_unhandled_error)
213
210
  self.finished_at = Time.current
214
-
215
- if GoodJob.preserve_job_records
216
- save!
217
- else
218
- destroy!
219
- end
211
+ save!
212
+ else
213
+ destroy!
220
214
  end
221
215
 
222
- [result, error]
216
+ [result, job_error]
223
217
  end
224
218
 
225
219
  private
@@ -56,7 +56,7 @@ module GoodJob
56
56
  # @example Get the records that have a session awaiting a lock:
57
57
  # MyLockableRecord.joins_advisory_locks.where("pg_locks.granted = ?", false)
58
58
  scope :joins_advisory_locks, (lambda do
59
- join_sql = <<~SQL
59
+ join_sql = <<~SQL.squish
60
60
  LEFT JOIN pg_locks ON pg_locks.locktype = 'advisory'
61
61
  AND pg_locks.objsubid = 1
62
62
  AND pg_locks.classid = ('x' || substr(md5(:table_name || #{quoted_table_name}.#{quoted_primary_key}::text), 1, 16))::bit(32)::int
@@ -140,10 +140,10 @@ module GoodJob
140
140
  # all remaining locks).
141
141
  # @return [Boolean] whether the lock was acquired.
142
142
  def advisory_lock
143
- where_sql = <<~SQL
143
+ where_sql = <<~SQL.squish
144
144
  pg_try_advisory_lock(('x' || substr(md5(:table_name || :id::text), 1, 16))::bit(64)::bigint)
145
145
  SQL
146
- self.class.unscoped.where(where_sql, { table_name: self.class.table_name, id: send(self.class.primary_key) }).exists?
146
+ self.class.unscoped.exists?([where_sql, { table_name: self.class.table_name, id: send(self.class.primary_key) }])
147
147
  end
148
148
 
149
149
  # Releases an advisory lock on this record if it is locked by this database
@@ -151,10 +151,10 @@ module GoodJob
151
151
  # {#advisory_unlock} and {#advisory_lock} the same number of times.
152
152
  # @return [Boolean] whether the lock was released.
153
153
  def advisory_unlock
154
- where_sql = <<~SQL
154
+ where_sql = <<~SQL.squish
155
155
  pg_advisory_unlock(('x' || substr(md5(:table_name || :id::text), 1, 16))::bit(64)::bigint)
156
156
  SQL
157
- self.class.unscoped.where(where_sql, { table_name: self.class.table_name, id: send(self.class.primary_key) }).exists?
157
+ self.class.unscoped.exists?([where_sql, { table_name: self.class.table_name, id: send(self.class.primary_key) }])
158
158
  end
159
159
 
160
160
  # Acquires an advisory lock on this record or raises
@@ -191,13 +191,13 @@ module GoodJob
191
191
  # Tests whether this record has an advisory lock on it.
192
192
  # @return [Boolean]
193
193
  def advisory_locked?
194
- self.class.unscoped.advisory_locked.where(id: send(self.class.primary_key)).exists?
194
+ self.class.unscoped.advisory_locked.exists?(id: send(self.class.primary_key))
195
195
  end
196
196
 
197
197
  # Tests whether this record is locked by the current database session.
198
198
  # @return [Boolean]
199
199
  def owns_advisory_lock?
200
- self.class.unscoped.owns_advisory_locked.where(id: send(self.class.primary_key)).exists?
200
+ self.class.unscoped.owns_advisory_locked.exists?(id: send(self.class.primary_key))
201
201
  end
202
202
 
203
203
  # Releases all advisory locks on the record that are held by the current
@@ -25,8 +25,7 @@ module GoodJob
25
25
  end
26
26
 
27
27
  # @macro notification_responder
28
- def timer_task_finished(event)
29
- # FIXME: This method does not match any good_job notifications.
28
+ def finished_timer_task(event)
30
29
  exception = event.payload[:error]
31
30
  return unless exception
32
31
 
@@ -36,8 +35,7 @@ module GoodJob
36
35
  end
37
36
 
38
37
  # @macro notification_responder
39
- def job_finished(event)
40
- # FIXME: This method does not match any good_job notifications.
38
+ def finished_job_task(event)
41
39
  exception = event.payload[:error]
42
40
  return unless exception
43
41
 
@@ -47,14 +45,13 @@ module GoodJob
47
45
  end
48
46
 
49
47
  # @macro notification_responder
50
- def scheduler_create_pools(event)
48
+ def scheduler_create_pool(event)
51
49
  max_threads = event.payload[:max_threads]
52
- poll_interval = event.payload[:poll_interval]
53
50
  performer_name = event.payload[:performer_name]
54
51
  process_id = event.payload[:process_id]
55
52
 
56
53
  info(tags: [process_id]) do
57
- "GoodJob started scheduler with queues=#{performer_name} max_threads=#{max_threads} poll_interval=#{poll_interval}."
54
+ "GoodJob started scheduler with queues=#{performer_name} max_threads=#{max_threads}."
58
55
  end
59
56
  end
60
57
 
@@ -168,12 +165,12 @@ module GoodJob
168
165
  # @return [Logger]
169
166
  def logger
170
167
  @_logger ||= begin
171
- logger = Logger.new(StringIO.new)
172
- loggers.each do |each_logger|
173
- logger.extend(ActiveSupport::Logger.broadcast(each_logger))
174
- end
175
- logger
176
- end
168
+ logger = Logger.new(StringIO.new)
169
+ loggers.each do |each_logger|
170
+ logger.extend(ActiveSupport::Logger.broadcast(each_logger))
171
+ end
172
+ logger
173
+ end
177
174
  end
178
175
 
179
176
  # Reset {LogSubscriber.logger} and force it to rebuild a new shortcut to
@@ -194,11 +191,12 @@ module GoodJob
194
191
  # @return [void]
195
192
  def tag_logger(*tags, &block)
196
193
  tags = tags.dup.unshift("GoodJob").compact
194
+ good_job_tag = ["ActiveJob"].freeze
197
195
 
198
196
  self.class.loggers.inject(block) do |inner, each_logger|
199
197
  if each_logger.respond_to?(:tagged)
200
198
  tags_for_logger = if each_logger.formatter.current_tags.include?("ActiveJob")
201
- ["ActiveJob"] + tags
199
+ good_job_tag + tags
202
200
  else
203
201
  tags
204
202
  end
@@ -26,14 +26,23 @@ module GoodJob
26
26
  # Delegates to {Scheduler#create_thread}.
27
27
  def create_thread(state = nil)
28
28
  results = []
29
- any_true = schedulers.any? do |scheduler|
30
- scheduler.create_thread(state).tap { |result| results << result }
29
+
30
+ if state
31
+ schedulers.any? do |scheduler|
32
+ scheduler.create_thread(state).tap { |result| results << result }
33
+ end
34
+ else
35
+ schedulers.each do |scheduler|
36
+ results << scheduler.create_thread(state)
37
+ end
31
38
  end
32
39
 
33
- if any_true
40
+ if results.any?
34
41
  true
35
- else
36
- results.any? { |result| result == false } ? false : nil
42
+ elsif results.any? { |result| result == false }
43
+ false
44
+ else # rubocop:disable Style/EmptyElse
45
+ nil
37
46
  end
38
47
  end
39
48
  end
@@ -34,7 +34,7 @@ module GoodJob # :nodoc:
34
34
  # @param message [#to_json]
35
35
  def self.notify(message)
36
36
  connection = ActiveRecord::Base.connection
37
- connection.exec_query <<~SQL
37
+ connection.exec_query <<~SQL.squish
38
38
  NOTIFY #{CHANNEL}, #{connection.quote(message.to_json)}
39
39
  SQL
40
40
  end
@@ -75,7 +75,7 @@ module GoodJob # :nodoc:
75
75
  # If +wait+ is +true+, the notifier will wait for background thread to shutdown.
76
76
  # If +wait+ is +false+, this method will return immediately even though threads may still be running.
77
77
  # Use {#shutdown?} to determine whether threads have stopped.
78
- # @param wait [Boolean] Wait for actively executing jobs to finish
78
+ # @param wait [Boolean] Wait for actively executing threads to finish
79
79
  # @return [void]
80
80
  def shutdown(wait: true)
81
81
  return unless @pool.running?
@@ -147,7 +147,7 @@ module GoodJob # :nodoc:
147
147
  pg_conn.exec("SET application_name = #{pg_conn.escape_identifier(self.class.name)}")
148
148
  yield pg_conn
149
149
  ensure
150
- ar_conn.disconnect!
150
+ ar_conn&.disconnect!
151
151
  end
152
152
  end
153
153
  end
@@ -0,0 +1,94 @@
1
+ require 'concurrent/atomic/atomic_boolean'
2
+
3
+ module GoodJob # :nodoc:
4
+ #
5
+ # Pollers regularly wake up execution threads to check for new work.
6
+ #
7
+ class Poller
8
+ # Defaults for instance of Concurrent::TimerTask.
9
+ # The timer controls how and when sleeping threads check for new work.
10
+ DEFAULT_TIMER_OPTIONS = {
11
+ execution_interval: Configuration::DEFAULT_POLL_INTERVAL,
12
+ timeout_interval: 1,
13
+ run_now: true,
14
+ }.freeze
15
+
16
+ # @!attribute [r] instances
17
+ # @!scope class
18
+ # List of all instantiated Pollers in the current process.
19
+ # @return [array<GoodJob:Poller>]
20
+ cattr_reader :instances, default: [], instance_reader: false
21
+
22
+ def self.from_configuration(configuration)
23
+ GoodJob::Poller.new(poll_interval: configuration.poll_interval)
24
+ end
25
+
26
+ # List of recipients that will receive notifications.
27
+ # @return [Array<#call, Array(Object, Symbol)>]
28
+ attr_reader :recipients
29
+
30
+ # @param recipients [Array<#call, Array(Object, Symbol)>]
31
+ # @param poll_interval [Hash] number of seconds between polls
32
+ def initialize(*recipients, poll_interval: nil)
33
+ @recipients = Concurrent::Array.new(recipients)
34
+
35
+ @timer_options = DEFAULT_TIMER_OPTIONS.dup
36
+ @timer_options[:execution_interval] = poll_interval if poll_interval.present?
37
+
38
+ self.class.instances << self
39
+
40
+ create_pool
41
+ end
42
+
43
+ # Shut down the poller.
44
+ # If +wait+ is +true+, the poller will wait for background thread to shutdown.
45
+ # If +wait+ is +false+, this method will return immediately even though threads may still be running.
46
+ # Use {#shutdown?} to determine whether threads have stopped.
47
+ # @param wait [Boolean] Wait for actively executing threads to finish
48
+ # @return [void]
49
+ def shutdown(wait: true)
50
+ return unless @timer&.running?
51
+
52
+ @timer.shutdown
53
+ @timer.wait_for_termination if wait
54
+ end
55
+
56
+ # Tests whether the poller is shutdown.
57
+ # @return [true, false, nil]
58
+ def shutdown?
59
+ !@timer&.running?
60
+ end
61
+
62
+ # Restart the poller.
63
+ # When shutdown, start; or shutdown and start.
64
+ # @param wait [Boolean] Wait for background thread to finish
65
+ # @return [void]
66
+ def restart(wait: true)
67
+ shutdown(wait: wait)
68
+ create_pool
69
+ end
70
+
71
+ # Invoked on completion of TimerTask task.
72
+ # @!visibility private
73
+ # @return [void]
74
+ def timer_observer(time, executed_task, thread_error)
75
+ GoodJob.on_thread_error.call(thread_error) if thread_error && GoodJob.on_thread_error.respond_to?(:call)
76
+ instrument("finished_timer_task", { result: executed_task, error: thread_error, time: time })
77
+ end
78
+
79
+ private
80
+
81
+ def create_pool
82
+ return if @timer_options[:execution_interval] <= 0
83
+
84
+ @timer = Concurrent::TimerTask.new(@timer_options) do
85
+ recipients.each do |recipient|
86
+ target, method_name = recipient.is_a?(Array) ? recipient : [recipient, :call]
87
+ target.send(method_name)
88
+ end
89
+ end
90
+ @timer.add_observer(self, :timer_observer)
91
+ @timer.execute
92
+ end
93
+ end
94
+ end
@@ -14,20 +14,12 @@ module GoodJob # :nodoc:
14
14
  # The scheduler maintains an instance of +Concurrent::TimerTask+, which wakes sleeping threads and causes them to check whether the performer has new work.
15
15
  #
16
16
  class Scheduler
17
- # Defaults for instance of Concurrent::TimerTask.
18
- # The timer controls how and when sleeping threads check for new work.
19
- DEFAULT_TIMER_OPTIONS = {
20
- execution_interval: 1,
21
- timeout_interval: 1,
22
- run_now: true,
23
- }.freeze
24
-
25
17
  # Defaults for instance of Concurrent::ThreadPoolExecutor
26
18
  # The thread pool is where work is performed.
27
19
  DEFAULT_POOL_OPTIONS = {
28
20
  name: name,
29
21
  min_threads: 0,
30
- max_threads: Concurrent.processor_count,
22
+ max_threads: Configuration::DEFAULT_MAX_THREADS,
31
23
  auto_terminate: true,
32
24
  idletime: 60,
33
25
  max_queue: -1,
@@ -41,7 +33,6 @@ module GoodJob # :nodoc:
41
33
  cattr_reader :instances, default: [], instance_reader: false
42
34
 
43
35
  # Creates GoodJob::Scheduler(s) and Performers from a GoodJob::Configuration instance.
44
- # TODO: move this to GoodJob::Configuration
45
36
  # @param configuration [GoodJob::Configuration]
46
37
  # @return [GoodJob::Scheduler, GoodJob::MultiScheduler]
47
38
  def self.from_configuration(configuration)
@@ -53,7 +44,7 @@ module GoodJob # :nodoc:
53
44
  parsed = GoodJob::Job.queue_parser(queue_string)
54
45
  job_filter = proc do |state|
55
46
  if parsed[:exclude]
56
- !parsed[:exclude].include? state[:queue_name]
47
+ parsed[:exclude].exclude?(state[:queue_name])
57
48
  elsif parsed[:include]
58
49
  parsed[:include].include? state[:queue_name]
59
50
  else
@@ -62,14 +53,7 @@ module GoodJob # :nodoc:
62
53
  end
63
54
  job_performer = GoodJob::Performer.new(job_query, :perform_with_advisory_lock, name: queue_string, filter: job_filter)
64
55
 
65
- timer_options = {}
66
- timer_options[:execution_interval] = configuration.poll_interval
67
-
68
- pool_options = {
69
- max_threads: max_threads,
70
- }
71
-
72
- GoodJob::Scheduler.new(job_performer, timer_options: timer_options, pool_options: pool_options)
56
+ GoodJob::Scheduler.new(job_performer, max_threads: max_threads)
73
57
  end
74
58
 
75
59
  if schedulers.size > 1
@@ -80,21 +64,19 @@ module GoodJob # :nodoc:
80
64
  end
81
65
 
82
66
  # @param performer [GoodJob::Performer]
83
- # @param timer_options [Hash] Options to instantiate a Concurrent::TimerTask
84
- # @param pool_options [Hash] Options to instantiate a Concurrent::ThreadPoolExecutor
85
- def initialize(performer, timer_options: {}, pool_options: {})
86
- # TODO: Replace `timer_options` and `pool_options` with only `poll_interval` and `max_threads`
67
+ # @param max_threads [Numeric, nil] number of seconds between polls for jobs
68
+ def initialize(performer, max_threads: nil)
87
69
  raise ArgumentError, "Performer argument must implement #next" unless performer.respond_to?(:next)
88
70
 
89
71
  self.class.instances << self
90
72
 
91
73
  @performer = performer
92
- @pool_options = DEFAULT_POOL_OPTIONS.merge(pool_options)
93
- @timer_options = DEFAULT_TIMER_OPTIONS.merge(timer_options)
94
74
 
95
- @pool_options[:name] = "GoodJob::Scheduler(queues=#{@performer.name} max_threads=#{@pool_options[:max_threads]} poll_interval=#{@timer_options[:execution_interval]})"
75
+ @pool_options = DEFAULT_POOL_OPTIONS.dup
76
+ @pool_options[:max_threads] = max_threads if max_threads.present?
77
+ @pool_options[:name] = "GoodJob::Scheduler(queues=#{@performer.name} max_threads=#{@pool_options[:max_threads]})"
96
78
 
97
- create_pools
79
+ create_pool
98
80
  end
99
81
 
100
82
  # Shut down the scheduler.
@@ -105,28 +87,20 @@ module GoodJob # :nodoc:
105
87
  # @param wait [Boolean] Wait for actively executing jobs to finish
106
88
  # @return [void]
107
89
  def shutdown(wait: true)
108
- @_shutdown = true
90
+ return unless @pool&.running?
109
91
 
110
92
  instrument("scheduler_shutdown_start", { wait: wait })
111
93
  instrument("scheduler_shutdown", { wait: wait }) do
112
- if @timer&.running?
113
- @timer.shutdown
114
- @timer.wait_for_termination if wait
115
- # TODO: Should be killed if wait is not true
116
- end
117
-
118
- if @pool&.running?
119
- @pool.shutdown
120
- @pool.wait_for_termination if wait
121
- # TODO: Should be killed if wait is not true
122
- end
94
+ @pool.shutdown
95
+ @pool.wait_for_termination if wait
96
+ # TODO: Should be killed if wait is not true
123
97
  end
124
98
  end
125
99
 
126
100
  # Tests whether the scheduler is shutdown.
127
101
  # @return [true, false, nil]
128
102
  def shutdown?
129
- @_shutdown
103
+ !@pool&.running?
130
104
  end
131
105
 
132
106
  # Restart the Scheduler.
@@ -136,8 +110,7 @@ module GoodJob # :nodoc:
136
110
  def restart(wait: true)
137
111
  instrument("scheduler_restart_pools") do
138
112
  shutdown(wait: wait) unless shutdown?
139
- create_pools
140
- @_shutdown = false
113
+ create_pool
141
114
  end
142
115
  end
143
116
 
@@ -162,14 +135,6 @@ module GoodJob # :nodoc:
162
135
  true
163
136
  end
164
137
 
165
- # Invoked on completion of TimerTask task.
166
- # @!visibility private
167
- # @return [void]
168
- def timer_observer(time, executed_task, thread_error)
169
- GoodJob.on_thread_error.call(thread_error) if thread_error && GoodJob.on_thread_error.respond_to?(:call)
170
- instrument("finished_timer_task", { result: executed_task, error: thread_error, time: time })
171
- end
172
-
173
138
  # Invoked on completion of ThreadPoolExecutor task
174
139
  # @!visibility private
175
140
  # @return [void]
@@ -181,14 +146,9 @@ module GoodJob # :nodoc:
181
146
 
182
147
  private
183
148
 
184
- def create_pools
185
- instrument("scheduler_create_pools", { performer_name: @performer.name, max_threads: @pool_options[:max_threads], poll_interval: @timer_options[:execution_interval] }) do
149
+ def create_pool
150
+ instrument("scheduler_create_pool", { performer_name: @performer.name, max_threads: @pool_options[:max_threads] }) do
186
151
  @pool = ThreadPoolExecutor.new(@pool_options)
187
- next unless @timer_options[:execution_interval].positive?
188
-
189
- @timer = Concurrent::TimerTask.new(@timer_options) { create_thread }
190
- @timer.add_observer(self, :timer_observer)
191
- @timer.execute
192
152
  end
193
153
  end
194
154
 
@@ -201,20 +161,20 @@ module GoodJob # :nodoc:
201
161
 
202
162
  ActiveSupport::Notifications.instrument("#{name}.good_job", payload, &block)
203
163
  end
204
- end
205
164
 
206
- # Custom sub-class of +Concurrent::ThreadPoolExecutor+ to add additional worker status.
207
- # @private
208
- class ThreadPoolExecutor < Concurrent::ThreadPoolExecutor
209
- # Number of inactive threads available to execute tasks.
210
- # https://github.com/ruby-concurrency/concurrent-ruby/issues/684#issuecomment-427594437
211
- # @return [Integer]
212
- def ready_worker_count
213
- synchronize do
214
- workers_still_to_be_created = @max_length - @pool.length
215
- workers_created_but_waiting = @ready.length
216
-
217
- workers_still_to_be_created + workers_created_but_waiting
165
+ # Custom sub-class of +Concurrent::ThreadPoolExecutor+ to add additional worker status.
166
+ # @private
167
+ class ThreadPoolExecutor < Concurrent::ThreadPoolExecutor
168
+ # Number of inactive threads available to execute tasks.
169
+ # https://github.com/ruby-concurrency/concurrent-ruby/issues/684#issuecomment-427594437
170
+ # @return [Integer]
171
+ def ready_worker_count
172
+ synchronize do
173
+ workers_still_to_be_created = @max_length - @pool.length
174
+ workers_created_but_waiting = @ready.length
175
+
176
+ workers_still_to_be_created + workers_created_but_waiting
177
+ end
218
178
  end
219
179
  end
220
180
  end
@@ -1,4 +1,4 @@
1
1
  module GoodJob
2
2
  # GoodJob gem version.
3
- VERSION = '1.2.5'.freeze
3
+ VERSION = '1.3.3'.freeze
4
4
  end
metadata CHANGED
@@ -1,14 +1,14 @@
1
1
  --- !ruby/object:Gem::Specification
2
2
  name: good_job
3
3
  version: !ruby/object:Gem::Version
4
- version: 1.2.5
4
+ version: 1.3.3
5
5
  platform: ruby
6
6
  authors:
7
7
  - Ben Sheldon
8
8
  autorequire:
9
9
  bindir: exe
10
10
  cert_chain: []
11
- date: 2020-09-17 00:00:00.000000000 Z
11
+ date: 2020-12-01 00:00:00.000000000 Z
12
12
  dependencies:
13
13
  - !ruby/object:Gem::Dependency
14
14
  name: activejob
@@ -290,6 +290,20 @@ dependencies:
290
290
  - - ">="
291
291
  - !ruby/object:Gem::Version
292
292
  version: '0'
293
+ - !ruby/object:Gem::Dependency
294
+ name: rails
295
+ requirement: !ruby/object:Gem::Requirement
296
+ requirements:
297
+ - - ">="
298
+ - !ruby/object:Gem::Version
299
+ version: '0'
300
+ type: :development
301
+ prerelease: false
302
+ version_requirements: !ruby/object:Gem::Requirement
303
+ requirements:
304
+ - - ">="
305
+ - !ruby/object:Gem::Version
306
+ version: '0'
293
307
  - !ruby/object:Gem::Dependency
294
308
  name: rbtrace
295
309
  requirement: !ruby/object:Gem::Requirement
@@ -475,6 +489,7 @@ files:
475
489
  - lib/good_job/multi_scheduler.rb
476
490
  - lib/good_job/notifier.rb
477
491
  - lib/good_job/performer.rb
492
+ - lib/good_job/poller.rb
478
493
  - lib/good_job/railtie.rb
479
494
  - lib/good_job/scheduler.rb
480
495
  - lib/good_job/version.rb
@@ -510,7 +525,7 @@ required_rubygems_version: !ruby/object:Gem::Requirement
510
525
  - !ruby/object:Gem::Version
511
526
  version: '0'
512
527
  requirements: []
513
- rubygems_version: 3.0.3
528
+ rubygems_version: 3.1.4
514
529
  signing_key:
515
530
  specification_version: 4
516
531
  summary: A multithreaded, Postgres-based ActiveJob backend for Ruby on Rails