good_job 1.4.1 → 1.8.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (21) hide show
  1. checksums.yaml +4 -4
  2. data/CHANGELOG.md +88 -7
  3. data/README.md +61 -15
  4. data/exe/good_job +1 -0
  5. data/lib/active_job/queue_adapters/good_job_adapter.rb +3 -3
  6. data/lib/good_job.rb +34 -7
  7. data/lib/good_job/adapter.rb +57 -33
  8. data/lib/good_job/cli.rb +25 -4
  9. data/lib/good_job/configuration.rb +76 -20
  10. data/lib/good_job/daemon.rb +59 -0
  11. data/lib/good_job/job.rb +25 -0
  12. data/lib/good_job/job_performer.rb +74 -0
  13. data/lib/good_job/lockable.rb +2 -2
  14. data/lib/good_job/multi_scheduler.rb +11 -6
  15. data/lib/good_job/notifier.rb +41 -32
  16. data/lib/good_job/poller.rb +31 -20
  17. data/lib/good_job/railtie.rb +10 -2
  18. data/lib/good_job/scheduler.rb +141 -66
  19. data/lib/good_job/version.rb +1 -1
  20. metadata +5 -4
  21. data/lib/good_job/performer.rb +0 -60
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: ac71f85784bd491d995252ed498d55642ee511a402ad093cc056fd2e062a437b
4
- data.tar.gz: 33f1a9ebf6952f3140046338bb616a4a81c469b4099f526e56d15b9aa1f8951d
3
+ metadata.gz: ff879dc7996d0f02055d1b61cff803a6ba24a5e0ec5441b8c81e3b7b1730fe63
4
+ data.tar.gz: 988e726ed6b8ba20d202bc86d7a21645ac8d08d2cf32b86aad258cfacf92e7db
5
5
  SHA512:
6
- metadata.gz: 1660dd44eca81ce42cbdb917253dfb340d5e5a4b371275f51eb919fc5aacb3485c6bbd87164d49d07547682efe311fb1a41bce35599ab5d45b8bd669d1e840c2
7
- data.tar.gz: 1c2994d95046ea9f692539dffcb0bfa33d81daed73886a2fef7ab9c9c6063b4085c17f8cc7ed93bcfab999ca25ffef1db31fa82344e3a31a6dcbdc3a84c966ed
6
+ metadata.gz: f268aa3249af67cad76da907afaa698726ab8b0ead6f004af1f13cd5fcf6e93b5f39b34540b3207252eaaac107f5f5d59bf0277ad76417a4ad7bd08ee811aaaf
7
+ data.tar.gz: 464a66b3dee0bfd771a5b28527cff4dffd0e704a9a301effc40ebec8ebaf1702b8180c0b97b2d78939913b1439f22aa25f9c434f0ae9ae5c88b136d07b4dd094
data/CHANGELOG.md CHANGED
@@ -1,5 +1,89 @@
1
1
  # Changelog
2
2
 
3
+ ## [v1.8.0](https://github.com/bensheldon/good_job/tree/v1.8.0) (2021-03-03)
4
+
5
+ [Full Changelog](https://github.com/bensheldon/good_job/compare/v1.7.1...v1.8.0)
6
+
7
+ **Implemented enhancements:**
8
+
9
+ - Wait then stop on shutdown [\#126](https://github.com/bensheldon/good_job/issues/126)
10
+ - Add shutdown-timeout option to configure the wait for jobs to gracefully finish before stopping them [\#213](https://github.com/bensheldon/good_job/pull/213) ([bensheldon](https://github.com/bensheldon))
11
+
12
+ **Fixed bugs:**
13
+
14
+ - Ensure Job\#serialized\_params are immutable [\#218](https://github.com/bensheldon/good_job/pull/218) ([bensheldon](https://github.com/bensheldon))
15
+
16
+ **Closed issues:**
17
+
18
+ - Run GoodJob on puma boot [\#91](https://github.com/bensheldon/good_job/issues/91)
19
+ - ActiveRecord::ConnectionNotEstablished when using async mode [\#89](https://github.com/bensheldon/good_job/issues/89)
20
+
21
+ **Merged pull requests:**
22
+
23
+ - Update bundler and Appraisals so Rails HEAD is locked to Ruby version \>= 2.7 [\#219](https://github.com/bensheldon/good_job/pull/219) ([bensheldon](https://github.com/bensheldon))
24
+
25
+ ## [v1.7.1](https://github.com/bensheldon/good_job/tree/v1.7.1) (2021-01-27)
26
+
27
+ [Full Changelog](https://github.com/bensheldon/good_job/compare/v1.7.0...v1.7.1)
28
+
29
+ **Fixed bugs:**
30
+
31
+ - Scheduler should always push a new task on completion of previous task, regardless of available thread calculation [\#209](https://github.com/bensheldon/good_job/pull/209) ([bensheldon](https://github.com/bensheldon))
32
+
33
+ **Closed issues:**
34
+
35
+ - Unexpected behavior with max\_threads = 1 [\#208](https://github.com/bensheldon/good_job/issues/208)
36
+
37
+ **Merged pull requests:**
38
+
39
+ - Fix equality typo in development.rb of test\_app [\#207](https://github.com/bensheldon/good_job/pull/207) ([reczy](https://github.com/reczy))
40
+
41
+ ## [v1.7.0](https://github.com/bensheldon/good_job/tree/v1.7.0) (2021-01-25)
42
+
43
+ [Full Changelog](https://github.com/bensheldon/good_job/compare/v1.6.0...v1.7.0)
44
+
45
+ **Implemented enhancements:**
46
+
47
+ - Cache scheduled jobs in memory so they can be executed without polling [\#205](https://github.com/bensheldon/good_job/pull/205) ([bensheldon](https://github.com/bensheldon))
48
+
49
+ ## [v1.6.0](https://github.com/bensheldon/good_job/tree/v1.6.0) (2021-01-22)
50
+
51
+ [Full Changelog](https://github.com/bensheldon/good_job/compare/v1.5.0...v1.6.0)
52
+
53
+ **Implemented enhancements:**
54
+
55
+ - Running as a daemon [\#88](https://github.com/bensheldon/good_job/issues/88)
56
+ - Add daemonize option to CLI [\#202](https://github.com/bensheldon/good_job/pull/202) ([bensheldon](https://github.com/bensheldon))
57
+
58
+ **Closed issues:**
59
+
60
+ - Rails 6.1 & async - `queue\_parser': undefined method `first' for "\*":String \(NoMethodError\) [\#195](https://github.com/bensheldon/good_job/issues/195)
61
+
62
+ **Merged pull requests:**
63
+
64
+ - Add scripts directory for benchmarking and dev tasks [\#204](https://github.com/bensheldon/good_job/pull/204) ([bensheldon](https://github.com/bensheldon))
65
+ - Fix YARD attr\_ declarations for documentation [\#203](https://github.com/bensheldon/good_job/pull/203) ([bensheldon](https://github.com/bensheldon))
66
+ - Remove Appraisal gemfile locks [\#201](https://github.com/bensheldon/good_job/pull/201) ([bensheldon](https://github.com/bensheldon))
67
+
68
+ ## [v1.5.0](https://github.com/bensheldon/good_job/tree/v1.5.0) (2021-01-18)
69
+
70
+ [Full Changelog](https://github.com/bensheldon/good_job/compare/v1.4.1...v1.5.0)
71
+
72
+ **Implemented enhancements:**
73
+
74
+ - Create Web UI Dashboard [\#50](https://github.com/bensheldon/good_job/issues/50)
75
+ - Configure GoodJob via `Rails.application.config` instead of recommending `GoodJob::Adapter.new` [\#199](https://github.com/bensheldon/good_job/pull/199) ([bensheldon](https://github.com/bensheldon))
76
+
77
+ **Closed issues:**
78
+
79
+ - JRuby Support [\#160](https://github.com/bensheldon/good_job/issues/160)
80
+
81
+ **Merged pull requests:**
82
+
83
+ - Update bundler version to 2.2.5 [\#200](https://github.com/bensheldon/good_job/pull/200) ([bensheldon](https://github.com/bensheldon))
84
+ - Update GH Test Matrix with minimum & latest JRuby version [\#197](https://github.com/bensheldon/good_job/pull/197) ([tedhexaflow](https://github.com/tedhexaflow))
85
+ - Fix JRuby version number [\#193](https://github.com/bensheldon/good_job/pull/193) ([tedhexaflow](https://github.com/tedhexaflow))
86
+
3
87
  ## [v1.4.1](https://github.com/bensheldon/good_job/tree/v1.4.1) (2021-01-09)
4
88
 
5
89
  [Full Changelog](https://github.com/bensheldon/good_job/compare/v1.4.0...v1.4.1)
@@ -15,6 +99,7 @@
15
99
  **Merged pull requests:**
16
100
 
17
101
  - Add missing YARD docs and Dashboard screenshot [\#191](https://github.com/bensheldon/good_job/pull/191) ([bensheldon](https://github.com/bensheldon))
102
+ - Update all Lockable queries to use exec\_query instead of execute; clear async\_exec results [\#189](https://github.com/bensheldon/good_job/pull/189) ([bensheldon](https://github.com/bensheldon))
18
103
 
19
104
  ## [v1.4.0](https://github.com/bensheldon/good_job/tree/v1.4.0) (2020-12-31)
20
105
 
@@ -41,7 +126,6 @@
41
126
  **Merged pull requests:**
42
127
 
43
128
  - Run tests with Rails default configuration to enable Zeitwerk [\#190](https://github.com/bensheldon/good_job/pull/190) ([bensheldon](https://github.com/bensheldon))
44
- - Update all Lockable queries to use exec\_query instead of execute; clear async\_exec results [\#189](https://github.com/bensheldon/good_job/pull/189) ([bensheldon](https://github.com/bensheldon))
45
129
  - Have Lockable\#advisory\_locked? directly query pg\_locks table [\#188](https://github.com/bensheldon/good_job/pull/188) ([bensheldon](https://github.com/bensheldon))
46
130
  - Update development gems, including Rails v6.1 and Rails HEAD [\#186](https://github.com/bensheldon/good_job/pull/186) ([bensheldon](https://github.com/bensheldon))
47
131
  - Update Appraisals for Rails 6.1 [\#183](https://github.com/bensheldon/good_job/pull/183) ([bensheldon](https://github.com/bensheldon))
@@ -212,7 +296,6 @@
212
296
  - Have YARD render markdown files with GFM \(Github Flavored Markdown\) [\#113](https://github.com/bensheldon/good_job/pull/113) ([bensheldon](https://github.com/bensheldon))
213
297
  - Add markdownlint to lint readme [\#109](https://github.com/bensheldon/good_job/pull/109) ([bensheldon](https://github.com/bensheldon))
214
298
  - Remove unused method in PgLocks [\#107](https://github.com/bensheldon/good_job/pull/107) ([gadimbaylisahil](https://github.com/gadimbaylisahil))
215
- - Re-organize Readme: frontload configuration, add Table of Contents [\#106](https://github.com/bensheldon/good_job/pull/106) ([bensheldon](https://github.com/bensheldon))
216
299
 
217
300
  ## [v1.2.3](https://github.com/bensheldon/good_job/tree/v1.2.3) (2020-08-27)
218
301
 
@@ -247,6 +330,7 @@
247
330
 
248
331
  **Merged pull requests:**
249
332
 
333
+ - Re-organize Readme: frontload configuration, add Table of Contents [\#106](https://github.com/bensheldon/good_job/pull/106) ([bensheldon](https://github.com/bensheldon))
250
334
  - Use more ActiveRecord in Lockable and not connection.execute [\#102](https://github.com/bensheldon/good_job/pull/102) ([bensheldon](https://github.com/bensheldon))
251
335
  - Run CI tests on Ruby 2.5, 2.6, and 2.7 [\#101](https://github.com/bensheldon/good_job/pull/101) ([arku](https://github.com/arku))
252
336
  - Fix Ruby 2.7 keyword arguments warning [\#98](https://github.com/bensheldon/good_job/pull/98) ([arku](https://github.com/arku))
@@ -379,6 +463,7 @@
379
463
 
380
464
  - Add migration generator [\#56](https://github.com/bensheldon/good_job/pull/56) ([thedanbob](https://github.com/thedanbob))
381
465
  - Fix migration script in readme [\#55](https://github.com/bensheldon/good_job/pull/55) ([thedanbob](https://github.com/thedanbob))
466
+ - Move where\(scheduled\_at: Time.current\) into dynamic part of GoodJob::Job::Performer [\#42](https://github.com/bensheldon/good_job/pull/42) ([bensheldon](https://github.com/bensheldon))
382
467
 
383
468
  ## [v1.0.1](https://github.com/bensheldon/good_job/tree/v1.0.1) (2020-07-22)
384
469
 
@@ -417,10 +502,6 @@
417
502
 
418
503
  [Full Changelog](https://github.com/bensheldon/good_job/compare/v0.8.0...v0.8.1)
419
504
 
420
- **Merged pull requests:**
421
-
422
- - Move where\(scheduled\_at: Time.current\) into dynamic part of GoodJob::Job::Performer [\#42](https://github.com/bensheldon/good_job/pull/42) ([bensheldon](https://github.com/bensheldon))
423
-
424
505
  ## [v0.8.0](https://github.com/bensheldon/good_job/tree/v0.8.0) (2020-07-17)
425
506
 
426
507
  [Full Changelog](https://github.com/bensheldon/good_job/compare/v0.7.0...v0.8.0)
@@ -508,7 +589,6 @@
508
589
  - Add pg gem as explicit dependency [\#13](https://github.com/bensheldon/good_job/pull/13) ([bensheldon](https://github.com/bensheldon))
509
590
  - Bump nokogiri from 1.10.7 to 1.10.9 [\#12](https://github.com/bensheldon/good_job/pull/12) ([dependabot[bot]](https://github.com/apps/dependabot))
510
591
  - Add Appraisal with tests for Rails 5.1, 5.2, 6.0 [\#11](https://github.com/bensheldon/good_job/pull/11) ([bensheldon](https://github.com/bensheldon))
511
- - Use Rails.logger and ActiveSupport::Notifications for logging instead of puts [\#10](https://github.com/bensheldon/good_job/pull/10) ([bensheldon](https://github.com/bensheldon))
512
592
 
513
593
  ## [v0.2.0](https://github.com/bensheldon/good_job/tree/v0.2.0) (2020-03-06)
514
594
 
@@ -516,6 +596,7 @@
516
596
 
517
597
  **Merged pull requests:**
518
598
 
599
+ - Use Rails.logger and ActiveSupport::Notifications for logging instead of puts [\#10](https://github.com/bensheldon/good_job/pull/10) ([bensheldon](https://github.com/bensheldon))
519
600
  - Remove minitest files [\#9](https://github.com/bensheldon/good_job/pull/9) ([bensheldon](https://github.com/bensheldon))
520
601
  - Use scheduled\_at and priority for scheduling [\#8](https://github.com/bensheldon/good_job/pull/8) ([bensheldon](https://github.com/bensheldon))
521
602
  - Create Github Action workflow for PRs and Issues [\#7](https://github.com/bensheldon/good_job/pull/7) ([bensheldon](https://github.com/bensheldon))
data/README.md CHANGED
@@ -35,8 +35,8 @@ For more of the story of GoodJob, read the [introductory blog post](https://isla
35
35
  - [Command-line options](#command-line-options)
36
36
  - [`good_job start`](#good_job-start)
37
37
  - [`good_job cleanup_preserved_jobs`](#good_job-cleanup_preserved_jobs)
38
- - [Adapter options](#adapter-options)
39
- - [Global options](#global-options)
38
+ - [Configuration options](#configuration-options)
39
+ - [Global options](#global-options)pter
40
40
  - [Dashboard](#dashboard)
41
41
  - [Go deeper](#go-deeper)
42
42
  - [Exceptions, retries, and reliability](#exceptions-retries-and-reliability)
@@ -127,7 +127,7 @@ For more of the story of GoodJob, read the [introductory blog post](https://isla
127
127
  ## Compatibility
128
128
 
129
129
  - **Ruby on Rails:** 5.2+
130
- - **Ruby:** MRI 2.5+. JRuby 9.13+ (_JRuby's `activerecord-jdbcpostgresql-adapter` gem does not support Postgres LISTEN/NOTIFY)._
130
+ - **Ruby:** MRI 2.5+. JRuby 9.2.13+ (_JRuby's `activerecord-jdbcpostgresql-adapter` gem does not support Postgres LISTEN/NOTIFY)._
131
131
  - **Postgres:** 9.6+
132
132
 
133
133
  ## Configuration
@@ -149,9 +149,13 @@ Usage:
149
149
  good_job start
150
150
 
151
151
  Options:
152
- [--max-threads=COUNT] # Maximum number of threads to use for working jobs. (env var: GOOD_JOB_MAX_THREADS, default: 5)
153
- [--queues=QUEUE_LIST] # Queues to work from. (env var: GOOD_JOB_QUEUES, default: *)
154
- [--poll-interval=SECONDS] # Interval between polls for available jobs in seconds (env var: GOOD_JOB_POLL_INTERVAL, default: 1)
152
+ [--max-threads=COUNT] # Maximum number of threads to use for working jobs. (env var: GOOD_JOB_MAX_THREADS, default: 5)
153
+ [--queues=QUEUE_LIST] # Queues to work from. (env var: GOOD_JOB_QUEUES, default: *)
154
+ [--poll-interval=SECONDS] # Interval between polls for available jobs in seconds (env var: GOOD_JOB_POLL_INTERVAL, default: 1)
155
+ [--max-cache=COUNT] # Maximum number of scheduled jobs to cache in memory (env var: GOOD_JOB_MAX_CACHE, default: 10000)
156
+ [--shutdown-timeout=SECONDS] # Number of seconds to wait for jobs to finish when shutting down before stopping the thread. (env var: GOOD_JOB_SHUTDOWN_TIMEOUT, default: -1 (forever))
157
+ [--daemonize] # Run as a background daemon (default: false)
158
+ [--pidfile=PIDFILE] # Path to write daemonized Process ID (env var: GOOD_JOB_PIDFILE, default: tmp/pids/good_job.pid)
155
159
 
156
160
  Executes queued jobs.
157
161
 
@@ -159,6 +163,7 @@ All options can be configured with environment variables.
159
163
  See option descriptions for the matching environment variable name.
160
164
 
161
165
  == Configuring queues
166
+
162
167
  Separate multiple queues with commas; exclude queues with a leading minus;
163
168
  separate isolated execution pools with semicolons and threads with colons.
164
169
  ```
@@ -189,9 +194,34 @@ If you are preserving job records this way, use this command regularly
189
194
  to delete old records and preserve space in your database.
190
195
  ```
191
196
 
192
- ### Adapter options
197
+ ### Configuration options
198
+
199
+ To use GoodJob, you can set `config.active_job.queue_adapter` to a `:good_job`.
200
+
201
+ Additional configuration can be provided via `config.good_job.OPTION = ...` for example:
202
+
203
+ ```ruby
204
+ # config/application.rb
205
+
206
+ config.active_job.queue_adapter = :good_job
207
+
208
+ # Configure options individually...
209
+ config.good_job.execution_mode = :async
210
+ config.good_job.max_threads = 5
211
+ config.good_job.poll_interval = 30 # seconds
212
+ config.good_job.shutdown_timeout = 25 # seconds
193
213
 
194
- To use GoodJob, you can set `config.active_job.queue_adapter` to a `:good_job` or to an instance of `GoodJob::Adapter`, which you can configure with several options:
214
+
215
+ # ...or all at once.
216
+ config.good_job = {
217
+ execution_mode: :async,
218
+ max_threads: 5,
219
+ poll_interval: 30,
220
+ shutdown_timeout: 25,
221
+ }
222
+ ```
223
+
224
+ Available configuration options are:
195
225
 
196
226
  - `execution_mode` (symbol) specifies how and where jobs should be executed. You can also set this with the environment variable `GOOD_JOB_EXECUTION_MODE`. It can be any one of:
197
227
  - `:inline` executes jobs immediately in whatever process queued them (usually the web server process). This should only be used in test and development environments.
@@ -200,18 +230,24 @@ To use GoodJob, you can set `config.active_job.queue_adapter` to a `:good_job` o
200
230
  - `max_threads` (integer) sets the maximum number of threads to use when `execution_mode` is set to `:async`. You can also set this with the environment variable `GOOD_JOB_MAX_THREADS`.
201
231
  - `queues` (string) determines which queues to execute jobs from when `execution_mode` is set to `:async`. See the description of `good_job start` for more details on the format of this string. You can also set this with the environment variable `GOOD_JOB_QUEUES`.
202
232
  - `poll_interval` (integer) sets the number of seconds between polls for jobs when `execution_mode` is set to `:async`. You can also set this with the environment variable `GOOD_JOB_POLL_INTERVAL`.
233
+ - `max_cache` (integer) sets the maximum number of scheduled jobs that will be stored in memory to reduce execution latency when also polling for scheduled jobs. Caching 10,000 scheduled jobs uses approximately 20MB of memory. You can also set this with the environment variable `GOOD_JOB_MAX_CACHE`.
234
+ - `shutdown_timeout` (float) number of seconds to wait for jobs to finish when shutting down before stopping the thread. Defaults to forever: `-1`. You can also set this with the environment variable `GOOD_JOB_SHUTDOWN_TIMEOUT`.
203
235
 
204
- Using the symbol instead of explicitly configuring the options above (i.e. setting `config.active_job.queue_adapter = :good_job`) is equivalent to:
236
+ By default, GoodJob configures the following execution modes per environment:
205
237
 
206
238
  ```ruby
239
+
207
240
  # config/environments/development.rb
208
- config.active_job.queue_adapter = GoodJob::Adapter.new(execution_mode: :inline)
241
+ config.active_job.queue_adapter = :good_job
242
+ config.good_job.execution_mode = :inline
209
243
 
210
244
  # config/environments/test.rb
211
- config.active_job.queue_adapter = GoodJob::Adapter.new(execution_mode: :inline)
245
+ config.active_job.queue_adapter = :good_job
246
+ config.good_job.execution_mode = :inline
212
247
 
213
248
  # config/environments/production.rb
214
- config.active_job.queue_adapter = GoodJob::Adapter.new(execution_mode: :external)
249
+ config.active_job.queue_adapter = :good_job
250
+ config.good_job.execution_mode = :external
215
251
  ```
216
252
 
217
253
  ### Global options
@@ -442,14 +478,24 @@ pool: <%= [ENV.fetch("RAILS_MAX_THREADS", 5).to_i, ENV.fetch("GOOD_JOB_MAX_THREA
442
478
 
443
479
  GoodJob can execute jobs "async" in the same process as the webserver (e.g. `bin/rail s`). GoodJob's async execution mode offers benefits of economy by not requiring a separate job worker process, but with the tradeoff of increased complexity. Async mode can be configured in two ways:
444
480
 
445
- - Directly configure the ActiveJob adapter:
481
+ - Via Rails configuration:
446
482
 
447
483
  ```ruby
448
484
  # config/environments/production.rb
449
- config.active_job.queue_adapter = GoodJob::Adapter.new(execution_mode: :async, max_threads: 4, poll_interval: 30)
485
+ config.active_job.queue_adapter = :good_job
486
+
487
+ # To change the execution mode
488
+ config.good_job.execution_mode = :async
489
+
490
+ # Or with more configuration
491
+ config.good_job = {
492
+ execution_mode: :async,
493
+ max_threads: 4,
494
+ poll_interval: 30
495
+ }
450
496
  ```
451
497
 
452
- - Or, when using `...queue_adapter = :good_job`, via environment variables:
498
+ - Or, with environment variables:
453
499
 
454
500
  ```bash
455
501
  $ GOOD_JOB_EXECUTION_MODE=async GOOD_JOB_MAX_THREADS=4 GOOD_JOB_POLL_INTERVAL=30 bin/rails server
data/exe/good_job CHANGED
@@ -1,4 +1,5 @@
1
1
  #!/usr/bin/env ruby
2
2
  require 'good_job/cli'
3
+ GOOD_JOB_WITHIN_CLI = true
3
4
  GOOD_JOB_LOG_TO_STDOUT = true
4
5
  GoodJob::CLI.start(ARGV)
data/lib/active_job/queue_adapters/good_job_adapter.rb CHANGED
@@ -2,9 +2,9 @@ module ActiveJob # :nodoc:
2
2
  module QueueAdapters # :nodoc:
3
3
  # See {GoodJob::Adapter} for details.
4
4
  class GoodJobAdapter < GoodJob::Adapter
5
- def initialize(execution_mode: nil, max_threads: nil, poll_interval: nil, scheduler: nil, inline: false)
6
- configuration = GoodJob::Configuration.new({ execution_mode: execution_mode }, env: ENV)
7
- super(execution_mode: configuration.rails_execution_mode, max_threads: max_threads, poll_interval: poll_interval, scheduler: scheduler, inline: inline)
5
+ def initialize(**options)
6
+ configuration = GoodJob::Configuration.new(options, env: ENV)
7
+ super(**options.merge(execution_mode: configuration.rails_execution_mode))
8
8
  end
9
9
  end
10
10
  end
data/lib/good_job.rb CHANGED
@@ -78,15 +78,26 @@ module GoodJob
78
78
  # See the {file:README.md#executing-jobs-async--in-process} for more explanation and examples.
79
79
  # @param wait [Boolean] whether to wait for shutdown
80
80
  # @return [void]
81
- def self.shutdown(wait: true)
82
- Notifier.instances.each { |notifier| notifier.shutdown(wait: wait) }
83
- Scheduler.instances.each { |scheduler| scheduler.shutdown(wait: wait) }
81
+ def self.shutdown(timeout: -1, wait: nil)
82
+ timeout = if wait.present?
83
+ ActiveSupport::Deprecation.warn(
84
+ "Using `GoodJob.shutdown` with `wait:` kwarg is deprecated; use `timeout:` kwarg instead e.g. GoodJob.shutdown(timeout: #{wait ? '-1' : 'nil'})"
85
+ )
86
+ wait ? -1 : nil
87
+ else
88
+ timeout
89
+ end
90
+
91
+ executables = Array(Notifier.instances) + Array(Poller.instances) + Array(Scheduler.instances)
92
+ _shutdown_all(executables, timeout: timeout)
84
93
  end
85
94
 
86
95
  # Tests whether jobs have stopped executing.
87
96
  # @return [Boolean] whether background threads are shut down
88
97
  def self.shutdown?
89
- Notifier.instances.all?(&:shutdown?) && Scheduler.instances.all?(&:shutdown?)
98
+ Notifier.instances.all?(&:shutdown?) &&
99
+ Poller.instances.all?(&:shutdown?) &&
100
+ Scheduler.instances.all?(&:shutdown?)
90
101
  end
91
102
 
92
103
  # Stops and restarts executing jobs.
@@ -95,9 +106,25 @@ module GoodJob
95
106
  # For example, you should use +shutdown+ and +restart+ when using async execution mode with Puma.
96
107
  # See the {file:README.md#executing-jobs-async--in-process} for more explanation and examples.
97
108
  # @return [void]
98
- def self.restart
99
- Notifier.instances.each(&:restart)
100
- Scheduler.instances.each(&:restart)
109
+ def self.restart(timeout: -1)
110
+ executables = Array(Notifier.instances) + Array(Poller.instances) + Array(Scheduler.instances)
111
+ _shutdown_all(executables, :restart, timeout: timeout)
112
+ end
113
+
114
+ # Sends +#shutdown+ or +#restart+ to executable objects ({GoodJob::Notifier}, {GoodJob::Poller}, {GoodJob::Scheduler})
115
+ # @param executables [Array<(Notifier, Poller, Scheduler)>] Objects to shut down.
116
+ # @param method_name [:symbol] Method to call, e.g. +:shutdown+ or +:restart+.
117
+ # @param timeout [nil,Numeric]
118
+ # @return [void]
119
+ def self._shutdown_all(executables, method_name = :shutdown, timeout: -1)
120
+ if timeout.positive?
121
+ executables.each { |executable| executable.send(method_name, timeout: nil) }
122
+
123
+ stop_at = Time.current + timeout
124
+ executables.each { |executable| executable.send(method_name, timeout: [stop_at - Time.current, 0].max) }
125
+ else
126
+ executables.each { |executable| executable.send(method_name, timeout: timeout) }
127
+ end
101
128
  end
102
129
 
103
130
  ActiveSupport.run_load_hooks(:good_job, self)
data/lib/good_job/adapter.rb CHANGED
@@ -20,16 +20,25 @@ module GoodJob
20
20
  # @param max_threads [nil, Integer] sets the number of threads per scheduler to use when +execution_mode+ is set to +:async+. The +queues+ parameter can specify a number of threads for each group of queues which will override this value. You can also set this with the environment variable +GOOD_JOB_MAX_THREADS+. Defaults to +5+.
21
21
  # @param queues [nil, String] determines which queues to execute jobs from when +execution_mode+ is set to +:async+. See {file:README.md#optimize-queues-threads-and-processes} for more details on the format of this string. You can also set this with the environment variable +GOOD_JOB_QUEUES+. Defaults to +"*"+.
22
22
  # @param poll_interval [nil, Integer] sets the number of seconds between polls for jobs when +execution_mode+ is set to +:async+. You can also set this with the environment variable +GOOD_JOB_POLL_INTERVAL+. Defaults to +1+.
23
- # @param scheduler [nil, Scheduler] (deprecated) a scheduler to be managed by the adapter
24
- # @param notifier [nil, Notifier] (deprecated) a notifier to be managed by the adapter
25
- # @param inline [nil, Boolean] (deprecated) whether to run in inline execution mode
26
- def initialize(execution_mode: nil, queues: nil, max_threads: nil, poll_interval: nil, scheduler: nil, notifier: nil, inline: false)
27
- if inline && execution_mode.nil?
28
- ActiveSupport::Deprecation.warn('GoodJob::Adapter#new(inline: true) is deprecated; use GoodJob::Adapter.new(execution_mode: :inline) instead')
29
- execution_mode = :inline
23
+ def initialize(execution_mode: nil, queues: nil, max_threads: nil, poll_interval: nil)
24
+ if caller[0..4].find { |c| c.include?("/config/application.rb") || c.include?("/config/environments/") }
25
+ ActiveSupport::Deprecation.warn(<<~DEPRECATION)
26
+ GoodJob no longer recommends creating a GoodJob::Adapter instance:
27
+
28
+ config.active_job.queue_adapter = GoodJob::Adapter.new...
29
+
30
+ Instead, configure GoodJob through configuration:
31
+
32
+ config.active_job.queue_adapter = :good_job
33
+ config.good_job.execution_mode = :#{execution_mode}
34
+ config.good_job.max_threads = #{max_threads}
35
+ config.good_job.poll_interval = #{poll_interval}
36
+ # etc...
37
+
38
+ DEPRECATION
30
39
  end
31
40
 
32
- configuration = GoodJob::Configuration.new(
41
+ @configuration = GoodJob::Configuration.new(
33
42
  {
34
43
  execution_mode: execution_mode,
35
44
  queues: queues,
@@ -38,13 +47,12 @@ module GoodJob
38
47
  }
39
48
  )
40
49
 
41
- @execution_mode = configuration.execution_mode
42
- raise ArgumentError, "execution_mode: must be one of #{EXECUTION_MODES.join(', ')}." unless EXECUTION_MODES.include?(@execution_mode)
50
+ raise ArgumentError, "execution_mode: must be one of #{EXECUTION_MODES.join(', ')}." unless EXECUTION_MODES.include?(@configuration.execution_mode)
43
51
 
44
- if @execution_mode == :async # rubocop:disable Style/GuardClause
45
- @notifier = notifier || GoodJob::Notifier.new
46
- @poller = GoodJob::Poller.new(poll_interval: configuration.poll_interval)
47
- @scheduler = scheduler || GoodJob::Scheduler.from_configuration(configuration)
52
+ if execute_async? # rubocop:disable Style/GuardClause
53
+ @notifier = GoodJob::Notifier.new
54
+ @poller = GoodJob::Poller.new(poll_interval: @configuration.poll_interval)
55
+ @scheduler = GoodJob::Scheduler.from_configuration(@configuration, warm_cache_on_initialize: Rails.application.initialized?)
48
56
  @notifier.recipients << [@scheduler, :create_thread]
49
57
  @poller.recipients << [@scheduler, :create_thread]
50
58
  end
@@ -76,43 +84,59 @@ module GoodJob
76
84
  ensure
77
85
  good_job.advisory_unlock
78
86
  end
79
- end
87
+ else
88
+ job_state = { queue_name: good_job.queue_name }
89
+ job_state[:scheduled_at] = good_job.scheduled_at if good_job.scheduled_at
80
90
 
81
- executed_locally = execute_async? && @scheduler.create_thread(queue_name: good_job.queue_name)
82
- Notifier.notify(queue_name: good_job.queue_name) unless executed_locally
91
+ executed_locally = execute_async? && @scheduler.create_thread(job_state)
92
+ Notifier.notify(job_state) unless executed_locally
93
+ end
83
94
 
84
95
  good_job
85
96
  end
86
97
 
87
- # Gracefully stop processing jobs.
88
- # Waits for termination by default.
89
- # @param wait [Boolean] Whether to wait for shut down.
98
+ # Shut down the thread pool executors.
99
+ # @param timeout [nil, Numeric] Seconds to wait for active threads.
100
+ #
101
+ # * +nil+, the scheduler will trigger a shutdown but not wait for it to complete.
102
+ # * +-1+, the scheduler will wait until the shutdown is complete.
103
+ # * +0+, the scheduler will immediately shutdown and stop any threads.
104
+ # * A positive number will wait that many seconds before stopping any remaining active threads.
105
+ # @param wait [Boolean] Deprecated. Use +timeout:+ instead.
90
106
  # @return [void]
91
- def shutdown(wait: true)
92
- @notifier&.shutdown(wait: wait)
93
- @poller&.shutdown(wait: wait)
94
- @scheduler&.shutdown(wait: wait)
107
+ def shutdown(timeout: :default, wait: nil)
108
+ timeout = if wait.present?
109
+ ActiveSupport::Deprecation.warn(
110
+ "Using `GoodJob::Adapter.shutdown` with `wait:` kwarg is deprecated; use `timeout:` kwarg instead e.g. GoodJob::Adapter.shutdown(timeout: #{wait ? '-1' : 'nil'})"
111
+ )
112
+ wait ? -1 : nil
113
+ else
114
+ timeout
115
+ end
116
+
117
+ timeout = if timeout == :default
118
+ @configuration.shutdown_timeout
119
+ else
120
+ timeout
121
+ end
122
+
123
+ executables = [@notifier, @poller, @scheduler].compact
124
+ GoodJob._shutdown_all(executables, timeout: timeout)
95
125
  end
96
126
 
97
127
  # Whether in +:async+ execution mode.
98
128
  def execute_async?
99
- @execution_mode == :async
129
+ @configuration.execution_mode == :async
100
130
  end
101
131
 
102
132
  # Whether in +:external+ execution mode.
103
133
  def execute_externally?
104
- @execution_mode == :external
134
+ @configuration.execution_mode == :external
105
135
  end
106
136
 
107
137
  # Whether in +:inline+ execution mode.
108
138
  def execute_inline?
109
- @execution_mode == :inline
110
- end
111
-
112
- # (deprecated) Whether in +:inline+ execution mode.
113
- def inline?
114
- ActiveSupport::Deprecation.warn('GoodJob::Adapter::inline? is deprecated; use GoodJob::Adapter::execute_inline? instead')
115
- execute_inline?
139
+ @configuration.execution_mode == :inline
116
140
  end
117
141
  end
118
142
  end