good_job 1.7.1 → 1.9.3

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (19) hide show
  1. checksums.yaml +4 -4
  2. data/CHANGELOG.md +109 -11
  3. data/README.md +26 -18
  4. data/exe/good_job +3 -2
  5. data/lib/active_job/queue_adapters/good_job_adapter.rb +0 -4
  6. data/lib/good_job.rb +48 -7
  7. data/lib/good_job/adapter.rb +51 -18
  8. data/lib/good_job/cli.rb +23 -8
  9. data/lib/good_job/configuration.rb +46 -40
  10. data/lib/good_job/execution_result.rb +20 -0
  11. data/lib/good_job/job.rb +35 -37
  12. data/lib/good_job/job_performer.rb +4 -3
  13. data/lib/good_job/lockable.rb +2 -2
  14. data/lib/good_job/multi_scheduler.rb +12 -7
  15. data/lib/good_job/notifier.rb +43 -35
  16. data/lib/good_job/poller.rb +31 -21
  17. data/lib/good_job/scheduler.rb +108 -73
  18. data/lib/good_job/version.rb +1 -1
  19. metadata +4 -17
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: 86268c6767ba44783707ed768fe726b7a53592e46468ed49bd38d427ff4919af
4
- data.tar.gz: 5fa72a4aaec59a31ba437eac31f9a6785cdb548c6694bea332434baf58aaad98
3
+ metadata.gz: 94f031dce3515297d2fa26803582e006a5787a681e3e2e7d50da00acea09cc27
4
+ data.tar.gz: 4fb166ced6dc815a3784a98d8ec2ae80d3ba843783435c035fadf577c83192a7
5
5
  SHA512:
6
- metadata.gz: c63baa745a861101b1a65c7a2dd7f4ec603a50714d5952a7ad942f30b41543643e6b7e1f19b1373f280668f212c6e173c4aa90d7d3a0770a92c1e159ae30215b
7
- data.tar.gz: 7bf564c4cfc0804f1a32995214195736e9cbd8d8f7a5b45d3ffe7efecf3d43126d412594855d50c9637c646e82838112a4869a4732fa0fe5ba1542b558facd0f
6
+ metadata.gz: 294c5bc738d9cf2953be24c0e5e9b3a04890d4861fb75c6e6040bbd49d63da2abf953dac367f92173c49c4e032824d6547f4a6f005bf035086bbd624b2db8405
7
+ data.tar.gz: c12d8e513db1e6c1976aa4e6478005d803f440f2cf75037fee389f111aa07303e539cdcc998a622f49571b78de84261503ee5e71e86b61702ea02bc82cd3e882
data/CHANGELOG.md CHANGED
@@ -1,23 +1,118 @@
1
1
  # Changelog
2
2
 
3
+ ## [v1.9.3](https://github.com/bensheldon/good_job/tree/v1.9.3) (2021-05-10)
4
+
5
+ [Full Changelog](https://github.com/bensheldon/good_job/compare/v1.9.2...v1.9.3)
6
+
7
+ **Implemented enhancements:**
8
+
9
+ - Add async\_server detection for extensions of rack handler [\#246](https://github.com/bensheldon/good_job/pull/246) ([bensheldon](https://github.com/bensheldon))
10
+
11
+ **Closed issues:**
12
+
13
+ - Not processing unfinished jobs across server restarts using async\_server mode [\#244](https://github.com/bensheldon/good_job/issues/244)
14
+ - No connection pool for 'ActiveRecord::Base' found [\#236](https://github.com/bensheldon/good_job/issues/236)
15
+
16
+ ## [v1.9.2](https://github.com/bensheldon/good_job/tree/v1.9.2) (2021-05-10)
17
+
18
+ [Full Changelog](https://github.com/bensheldon/good_job/compare/v1.9.1...v1.9.2)
19
+
20
+ **Fixed bugs:**
21
+
22
+ - Run Scheduler\#warm\_cache operation in threadpool executor [\#242](https://github.com/bensheldon/good_job/pull/242) ([bensheldon](https://github.com/bensheldon))
23
+
24
+ **Closed issues:**
25
+
26
+ - Jobs not visible in dashboard [\#245](https://github.com/bensheldon/good_job/issues/245)
27
+
28
+ **Merged pull requests:**
29
+
30
+ - Use GoodJob::Job::ExecutionResult object instead of job execution returning an ordered array [\#241](https://github.com/bensheldon/good_job/pull/241) ([bensheldon](https://github.com/bensheldon))
31
+ - Update development dependencies [\#240](https://github.com/bensheldon/good_job/pull/240) ([bensheldon](https://github.com/bensheldon))
32
+
33
+ ## [v1.9.1](https://github.com/bensheldon/good_job/tree/v1.9.1) (2021-04-19)
34
+
35
+ [Full Changelog](https://github.com/bensheldon/good_job/compare/v1.9.0...v1.9.1)
36
+
37
+ **Implemented enhancements:**
38
+
39
+ - Allow to specify parent class for active record [\#238](https://github.com/bensheldon/good_job/pull/238) ([morgoth](https://github.com/morgoth))
40
+
41
+ ## [v1.9.0](https://github.com/bensheldon/good_job/tree/v1.9.0) (2021-04-16)
42
+
43
+ [Full Changelog](https://github.com/bensheldon/good_job/compare/v1.8.0...v1.9.0)
44
+
45
+ **Implemented enhancements:**
46
+
47
+ - Add `async_server` option to run async only in Rails web server process [\#230](https://github.com/bensheldon/good_job/pull/230) ([bensheldon](https://github.com/bensheldon))
48
+ - FreeBSD startup script [\#221](https://github.com/bensheldon/good_job/pull/221) ([lauer](https://github.com/lauer))
49
+
50
+ **Fixed bugs:**
51
+
52
+ - Fix instrumentation of GoodJob::Poller finished\_timer\_task event [\#233](https://github.com/bensheldon/good_job/pull/233) ([bensheldon](https://github.com/bensheldon))
53
+
54
+ **Closed issues:**
55
+
56
+ - Cannot run db:migrate when execution mode is :async [\#229](https://github.com/bensheldon/good_job/issues/229)
57
+ - How do you enqueue a job to be executed immediately outside of Rails \(eg. creating a new record of good\_jobs in Postgresql\)? [\#225](https://github.com/bensheldon/good_job/issues/225)
58
+ - Feature Ideas [\#220](https://github.com/bensheldon/good_job/issues/220)
59
+ - Goodjob startup script for FreeBSD [\#214](https://github.com/bensheldon/good_job/issues/214)
60
+ - Only start async mode executors when server is running [\#194](https://github.com/bensheldon/good_job/issues/194)
61
+
62
+ **Merged pull requests:**
63
+
64
+ - Move executable flags from constants to accessors on GoodJob::CLI [\#234](https://github.com/bensheldon/good_job/pull/234) ([bensheldon](https://github.com/bensheldon))
65
+ - Add custom Scheduler::TimerSet [\#232](https://github.com/bensheldon/good_job/pull/232) ([bensheldon](https://github.com/bensheldon))
66
+ - Fix assorted constant references in YARD documentation [\#231](https://github.com/bensheldon/good_job/pull/231) ([bensheldon](https://github.com/bensheldon))
67
+ - Update GH Test Matrix with latest JRuby 9.2.17.0 [\#228](https://github.com/bensheldon/good_job/pull/228) ([tedhexaflow](https://github.com/tedhexaflow))
68
+ - Update gem dependencies [\#227](https://github.com/bensheldon/good_job/pull/227) ([bensheldon](https://github.com/bensheldon))
69
+ - Remove leftover text from Readme [\#226](https://github.com/bensheldon/good_job/pull/226) ([weh](https://github.com/weh))
70
+ - Fix appraisal and bundler version CI conflicts [\#224](https://github.com/bensheldon/good_job/pull/224) ([bensheldon](https://github.com/bensheldon))
71
+ - Update GH Test Matrix with latest JRuby [\#223](https://github.com/bensheldon/good_job/pull/223) ([tedhexaflow](https://github.com/tedhexaflow))
72
+
73
+ ## [v1.8.0](https://github.com/bensheldon/good_job/tree/v1.8.0) (2021-03-04)
74
+
75
+ [Full Changelog](https://github.com/bensheldon/good_job/compare/v1.7.1...v1.8.0)
76
+
77
+ **Implemented enhancements:**
78
+
79
+ - Wait then stop on shutdown [\#126](https://github.com/bensheldon/good_job/issues/126)
80
+ - Add shutdown-timeout option to configure the wait for jobs to gracefully finish before stopping them [\#213](https://github.com/bensheldon/good_job/pull/213) ([bensheldon](https://github.com/bensheldon))
81
+
82
+ **Fixed bugs:**
83
+
84
+ - Ensure Job\#serialized\_params are immutable [\#218](https://github.com/bensheldon/good_job/pull/218) ([bensheldon](https://github.com/bensheldon))
85
+
86
+ **Closed issues:**
87
+
88
+ - Run GoodJob on puma boot [\#91](https://github.com/bensheldon/good_job/issues/91)
89
+ - ActiveRecord::ConnectionNotEstablished when using async mode [\#89](https://github.com/bensheldon/good_job/issues/89)
90
+
91
+ **Merged pull requests:**
92
+
93
+ - Update bundler and Appraisals so Rails HEAD is locked to Ruby version \>= 2.7 [\#219](https://github.com/bensheldon/good_job/pull/219) ([bensheldon](https://github.com/bensheldon))
94
+
3
95
  ## [v1.7.1](https://github.com/bensheldon/good_job/tree/v1.7.1) (2021-01-27)
4
96
 
5
97
  [Full Changelog](https://github.com/bensheldon/good_job/compare/v1.7.0...v1.7.1)
6
98
 
99
+ **Fixed bugs:**
100
+
101
+ - Scheduler should always push a new task on completion of previous task, regardless of available thread calculation [\#209](https://github.com/bensheldon/good_job/pull/209) ([bensheldon](https://github.com/bensheldon))
102
+
7
103
  **Closed issues:**
8
104
 
9
105
  - Unexpected behavior with max\_threads = 1 [\#208](https://github.com/bensheldon/good_job/issues/208)
10
106
 
11
107
  **Merged pull requests:**
12
108
 
13
- - Scheduler should always push a new task on completion of previous task, regardless of available thread calculation [\#209](https://github.com/bensheldon/good_job/pull/209) ([bensheldon](https://github.com/bensheldon))
14
109
  - Fix equality typo in development.rb of test\_app [\#207](https://github.com/bensheldon/good_job/pull/207) ([reczy](https://github.com/reczy))
15
110
 
16
111
  ## [v1.7.0](https://github.com/bensheldon/good_job/tree/v1.7.0) (2021-01-25)
17
112
 
18
113
  [Full Changelog](https://github.com/bensheldon/good_job/compare/v1.6.0...v1.7.0)
19
114
 
20
- **Merged pull requests:**
115
+ **Implemented enhancements:**
21
116
 
22
117
  - Cache scheduled jobs in memory so they can be executed without polling [\#205](https://github.com/bensheldon/good_job/pull/205) ([bensheldon](https://github.com/bensheldon))
23
118
 
@@ -32,7 +127,7 @@
32
127
 
33
128
  **Closed issues:**
34
129
 
35
- - Rails 6.1 & async - `queue\_parser': undefined method `first' for "\*":String \(NoMethodError\) [\#195](https://github.com/bensheldon/good_job/issues/195)
130
+ - Rails 6.1 & async - `queue_parser': undefined method `first' for "\*":String \(NoMethodError\) [\#195](https://github.com/bensheldon/good_job/issues/195)
36
131
 
37
132
  **Merged pull requests:**
38
133
 
@@ -47,6 +142,7 @@
47
142
  **Implemented enhancements:**
48
143
 
49
144
  - Create Web UI Dashboard [\#50](https://github.com/bensheldon/good_job/issues/50)
145
+ - Configure GoodJob via `Rails.application.config` instead of recommending `GoodJob::Adapter.new` [\#199](https://github.com/bensheldon/good_job/pull/199) ([bensheldon](https://github.com/bensheldon))
50
146
 
51
147
  **Closed issues:**
52
148
 
@@ -55,7 +151,6 @@
55
151
  **Merged pull requests:**
56
152
 
57
153
  - Update bundler version to 2.2.5 [\#200](https://github.com/bensheldon/good_job/pull/200) ([bensheldon](https://github.com/bensheldon))
58
- - Configure GoodJob via `Rails.application.config` instead of recommending `GoodJob::Adapter.new` [\#199](https://github.com/bensheldon/good_job/pull/199) ([bensheldon](https://github.com/bensheldon))
59
154
  - Update GH Test Matrix with minimum & latest JRuby version [\#197](https://github.com/bensheldon/good_job/pull/197) ([tedhexaflow](https://github.com/tedhexaflow))
60
155
  - Fix JRuby version number [\#193](https://github.com/bensheldon/good_job/pull/193) ([tedhexaflow](https://github.com/tedhexaflow))
61
156
 
@@ -114,7 +209,6 @@
114
209
 
115
210
  - Ensure advisory lock CTE is MATERIALIZED on Postgres v12+ [\#179](https://github.com/bensheldon/good_job/pull/179) ([bensheldon](https://github.com/bensheldon))
116
211
  - Ensure that deleted jobs are unlocked [\#178](https://github.com/bensheldon/good_job/pull/178) ([bensheldon](https://github.com/bensheldon))
117
- - Fix job ordering for Rails 6.1 [\#174](https://github.com/bensheldon/good_job/pull/174) ([morgoth](https://github.com/morgoth))
118
212
 
119
213
  **Closed issues:**
120
214
 
@@ -129,6 +223,10 @@
129
223
 
130
224
  [Full Changelog](https://github.com/bensheldon/good_job/compare/v1.3.3...v1.3.4)
131
225
 
226
+ **Fixed bugs:**
227
+
228
+ - Fix job ordering for Rails 6.1 [\#174](https://github.com/bensheldon/good_job/pull/174) ([morgoth](https://github.com/morgoth))
229
+
132
230
  ## [v1.3.3](https://github.com/bensheldon/good_job/tree/v1.3.3) (2020-12-01)
133
231
 
134
232
  [Full Changelog](https://github.com/bensheldon/good_job/compare/v1.3.2...v1.3.3)
@@ -189,7 +287,7 @@
189
287
  **Implemented enhancements:**
190
288
 
191
289
  - Preserve only failed jobs [\#136](https://github.com/bensheldon/good_job/issues/136)
192
- - Add `GoodJob.preserve\_job\_records = :on\_unhandled\_error` option to only preserve jobs that errored [\#145](https://github.com/bensheldon/good_job/pull/145) ([morgoth](https://github.com/morgoth))
290
+ - Add `GoodJob.preserve_job_records = :on_unhandled_error` option to only preserve jobs that errored [\#145](https://github.com/bensheldon/good_job/pull/145) ([morgoth](https://github.com/morgoth))
193
291
 
194
292
  **Fixed bugs:**
195
293
 
@@ -222,7 +320,7 @@
222
320
  - Update Gemspec to reflect that GoodJob is not compatible with Rails 5.1 [\#143](https://github.com/bensheldon/good_job/pull/143) ([bensheldon](https://github.com/bensheldon))
223
321
  - Prevent jobs hanging [\#141](https://github.com/bensheldon/good_job/pull/141) ([morgoth](https://github.com/morgoth))
224
322
  - Add explicit require\_paths to gemspec for engine [\#134](https://github.com/bensheldon/good_job/pull/134) ([bensheldon](https://github.com/bensheldon))
225
- - Use `connection.quote\_table\_name` and add spacing for SQL concatenation [\#124](https://github.com/bensheldon/good_job/pull/124) ([bensheldon](https://github.com/bensheldon))
323
+ - Use `connection.quote_table_name` and add spacing for SQL concatenation [\#124](https://github.com/bensheldon/good_job/pull/124) ([bensheldon](https://github.com/bensheldon))
226
324
 
227
325
  **Closed issues:**
228
326
 
@@ -248,7 +346,7 @@
248
346
 
249
347
  **Implemented enhancements:**
250
348
 
251
- - Add environment variable to mirror `cleanup\_preserved\_jobs --before-seconds-ago=SECONDS` [\#110](https://github.com/bensheldon/good_job/issues/110)
349
+ - Add environment variable to mirror `cleanup_preserved_jobs --before-seconds-ago=SECONDS` [\#110](https://github.com/bensheldon/good_job/issues/110)
252
350
  - Allow env variable config for cleanups [\#114](https://github.com/bensheldon/good_job/pull/114) ([gadimbaylisahil](https://github.com/gadimbaylisahil))
253
351
 
254
352
  **Fixed bugs:**
@@ -299,7 +397,7 @@
299
397
 
300
398
  **Closed issues:**
301
399
 
302
- - Add test for `rails g good\_job:install` [\#57](https://github.com/bensheldon/good_job/issues/57)
400
+ - Add test for `rails g good_job:install` [\#57](https://github.com/bensheldon/good_job/issues/57)
303
401
 
304
402
  **Merged pull requests:**
305
403
 
@@ -307,7 +405,7 @@
307
405
  - Run CI tests on Ruby 2.5, 2.6, and 2.7 [\#101](https://github.com/bensheldon/good_job/pull/101) ([arku](https://github.com/arku))
308
406
  - Fix Ruby 2.7 keyword arguments warning [\#98](https://github.com/bensheldon/good_job/pull/98) ([arku](https://github.com/arku))
309
407
  - Remove executor/reloader for less interlocking [\#97](https://github.com/bensheldon/good_job/pull/97) ([sj26](https://github.com/sj26))
310
- - Add test for `rails g good\_job:install` [\#94](https://github.com/bensheldon/good_job/pull/94) ([arku](https://github.com/arku))
408
+ - Add test for `rails g good_job:install` [\#94](https://github.com/bensheldon/good_job/pull/94) ([arku](https://github.com/arku))
311
409
 
312
410
  ## [v1.2.1](https://github.com/bensheldon/good_job/tree/v1.2.1) (2020-08-21)
313
411
 
@@ -514,7 +612,7 @@
514
612
  - Update Github Action Workflow for Backlog Project Board [\#35](https://github.com/bensheldon/good_job/pull/35) ([bensheldon](https://github.com/bensheldon))
515
613
  - Add configuration options to good\_job executable [\#33](https://github.com/bensheldon/good_job/pull/33) ([bensheldon](https://github.com/bensheldon))
516
614
  - Extract Job querying behavior out of Scheduler [\#31](https://github.com/bensheldon/good_job/pull/31) ([bensheldon](https://github.com/bensheldon))
517
- - Allow configuration of Rails queue adapter with `:good\_job` [\#28](https://github.com/bensheldon/good_job/pull/28) ([bensheldon](https://github.com/bensheldon))
615
+ - Allow configuration of Rails queue adapter with `:good_job` [\#28](https://github.com/bensheldon/good_job/pull/28) ([bensheldon](https://github.com/bensheldon))
518
616
 
519
617
  ## [v0.5.0](https://github.com/bensheldon/good_job/tree/v0.5.0) (2020-07-13)
520
618
 
data/README.md CHANGED
@@ -36,7 +36,7 @@ For more of the story of GoodJob, read the [introductory blog post](https://isla
36
36
  - [`good_job start`](#good_job-start)
37
37
  - [`good_job cleanup_preserved_jobs`](#good_job-cleanup_preserved_jobs)
38
38
  - [Configuration options](#configuration-options)
39
- - [Global options](#global-options)pter
39
+ - [Global options](#global-options)
40
40
  - [Dashboard](#dashboard)
41
41
  - [Go deeper](#go-deeper)
42
42
  - [Exceptions, retries, and reliability](#exceptions-retries-and-reliability)
@@ -119,10 +119,10 @@ For more of the story of GoodJob, read the [introductory blog post](https://isla
119
119
  - GoodJob can also be configured to execute jobs within the web server process to save on resources. This is useful for low-workloads when economy is paramount.
120
120
 
121
121
  ```
122
- $ GOOD_JOB_EXECUTION_MODE=async rails server
122
+ $ GOOD_JOB_EXECUTION_MODE=async_server rails server
123
123
  ```
124
124
 
125
- Additional configuration is likely necessary, see the reference below for async configuration.
125
+ Additional configuration is likely necessary, see the reference below for f configuration.
126
126
 
127
127
  ## Compatibility
128
128
 
@@ -149,12 +149,13 @@ Usage:
149
149
  good_job start
150
150
 
151
151
  Options:
152
- [--max-threads=COUNT] # Maximum number of threads to use for working jobs. (env var: GOOD_JOB_MAX_THREADS, default: 5)
153
- [--queues=QUEUE_LIST] # Queues to work from. (env var: GOOD_JOB_QUEUES, default: *)
154
- [--poll-interval=SECONDS] # Interval between polls for available jobs in seconds (env var: GOOD_JOB_POLL_INTERVAL, default: 1)
155
- [--max-cache=COUNT] # Maximum number of scheduled jobs to cache in memory (env var: GOOD_JOB_MAX_CACHE, default: 10000)
156
- [--daemonize] # Run as a background daemon (default: false)
157
- [--pidfile=PIDFILE] # Path to write daemonized Process ID (env var: GOOD_JOB_PIDFILE, default: tmp/pids/good_job.pid)
152
+ [--max-threads=COUNT] # Maximum number of threads to use for working jobs. (env var: GOOD_JOB_MAX_THREADS, default: 5)
153
+ [--queues=QUEUE_LIST] # Queues to work from. (env var: GOOD_JOB_QUEUES, default: *)
154
+ [--poll-interval=SECONDS] # Interval between polls for available jobs in seconds (env var: GOOD_JOB_POLL_INTERVAL, default: 1)
155
+ [--max-cache=COUNT] # Maximum number of scheduled jobs to cache in memory (env var: GOOD_JOB_MAX_CACHE, default: 10000)
156
+ [--shutdown-timeout=SECONDS] # Number of seconds to wait for jobs to finish when shutting down before stopping the thread. (env var: GOOD_JOB_SHUTDOWN_TIMEOUT, default: -1 (forever))
157
+ [--daemonize] # Run as a background daemon (default: false)
158
+ [--pidfile=PIDFILE] # Path to write daemonized Process ID (env var: GOOD_JOB_PIDFILE, default: tmp/pids/good_job.pid)
158
159
 
159
160
  Executes queued jobs.
160
161
 
@@ -205,15 +206,18 @@ Additional configuration can be provided via `config.good_job.OPTION = ...` for
205
206
  config.active_job.queue_adapter = :good_job
206
207
 
207
208
  # Configure options individually...
208
- config.good_job.execution_mode = :async
209
+ config.good_job.execution_mode = :async_server
209
210
  config.good_job.max_threads = 5
210
211
  config.good_job.poll_interval = 30 # seconds
212
+ config.good_job.shutdown_timeout = 25 # seconds
213
+
211
214
 
212
215
  # ...or all at once.
213
216
  config.good_job = {
214
- execution_mode: :async,
217
+ execution_mode: :async_server,
215
218
  max_threads: 5,
216
219
  poll_interval: 30,
220
+ shutdown_timeout: 25,
217
221
  }
218
222
  ```
219
223
 
@@ -222,11 +226,13 @@ Available configuration options are:
222
226
  - `execution_mode` (symbol) specifies how and where jobs should be executed. You can also set this with the environment variable `GOOD_JOB_EXECUTION_MODE`. It can be any one of:
223
227
  - `:inline` executes jobs immediately in whatever process queued them (usually the web server process). This should only be used in test and development environments.
224
228
  - `:external` causes the adapter to enqueue jobs, but not execute them. When using this option (the default for production environments), you’ll need to use the command-line tool to actually execute your jobs.
225
- - `:async` causes the adapter to execute you jobs in separate threads in whatever process queued them (usually the web process). This is akin to running the command-line tool’s code inside your web server. It can be more economical for small workloads (you don’t need a separate machine or environment for running your jobs), but if your web server is under heavy load or your jobs require a lot of resources, you should choose `:external` instead.
226
- - `max_threads` (integer) sets the maximum number of threads to use when `execution_mode` is set to `:async`. You can also set this with the environment variable `GOOD_JOB_MAX_THREADS`.
227
- - `queues` (string) determines which queues to execute jobs from when `execution_mode` is set to `:async`. See the description of `good_job start` for more details on the format of this string. You can also set this with the environment variable `GOOD_JOB_QUEUES`.
228
- - `poll_interval` (integer) sets the number of seconds between polls for jobs when `execution_mode` is set to `:async`. You can also set this with the environment variable `GOOD_JOB_POLL_INTERVAL`.
229
+ - `:async_server` executes jobs in separate threads within the Rails webserver process (`bundle exec rails server`). It can be more economical for small workloads because you don’t need a separate machine or environment for running your jobs, but if your web server is under heavy load or your jobs require a lot of resources, you should choose `:external` instead. When not in the Rails webserver, jobs will execute in `:external` mode to ensure jobs are not executed within `rails console`, `rails db:migrate`, `rails assets:prepare`, etc.
230
+ - `:async` executes jobs in separate threads in _any_ Rails process.
231
+ - `max_threads` (integer) sets the maximum number of threads to use when `execution_mode` is set to `:async` or `:async_server`. You can also set this with the environment variable `GOOD_JOB_MAX_THREADS`.
232
+ - `queues` (string) determines which queues to execute jobs from when `execution_mode` is set to `:async` or `:async_server`. See the description of `good_job start` for more details on the format of this string. You can also set this with the environment variable `GOOD_JOB_QUEUES`.
233
+ - `poll_interval` (integer) sets the number of seconds between polls for jobs when `execution_mode` is set to `:async` or `:async_server`. You can also set this with the environment variable `GOOD_JOB_POLL_INTERVAL`.
229
234
  - `max_cache` (integer) sets the maximum number of scheduled jobs that will be stored in memory to reduce execution latency when also polling for scheduled jobs. Caching 10,000 scheduled jobs uses approximately 20MB of memory. You can also set this with the environment variable `GOOD_JOB_MAX_CACHE`.
235
+ - `shutdown_timeout` (float) number of seconds to wait for jobs to finish when shutting down before stopping the thread. Defaults to forever: `-1`. You can also set this with the environment variable `GOOD_JOB_SHUTDOWN_TIMEOUT`.
230
236
 
231
237
  By default, GoodJob configures the following execution modes per environment:
232
238
 
@@ -249,6 +255,7 @@ config.good_job.execution_mode = :external
249
255
 
250
256
  Good Job’s general behavior can also be configured via several attributes directly on the `GoodJob` module:
251
257
 
258
+ - **`GoodJob.active_record_parent_class`** (string) The ActiveRecord parent class inherited by GoodJob's ActiveRecord model `GoodJob::Job` (defaults to `"ActiveRecord::Base"`). Configure this when using [multiple databases with ActiveRecord](https://guides.rubyonrails.org/active_record_multiple_databases.html) or when other custom configuration is necessary for the ActiveRecord model to connect to the Postgres database. _The value must be a String to avoid premature initialization of ActiveRecord._
252
259
  - **`GoodJob.logger`** ([Rails Logger](https://api.rubyonrails.org/classes/ActiveSupport/Logger.html)) lets you set a custom logger for GoodJob. It should be an instance of a Rails `Logger`.
253
260
  - **`GoodJob.preserve_job_records`** (boolean) keeps job records in your database even after jobs are completed. (Default: `false`)
254
261
  - **`GoodJob.retry_on_unhandled_error`** (boolean) causes jobs to be re-queued and retried if they raise an instance of `StandardError`. Instances of `Exception`, like SIGINT, will *always* be retried, regardless of this attribute’s value. (Default: `true`)
@@ -258,6 +265,7 @@ You’ll generally want to configure these in `config/initializers/good_job.rb`,
258
265
 
259
266
  ```ruby
260
267
  # config/initializers/good_job.rb
268
+ GoodJob.active_record_parent_class = "ApplicationRecord"
261
269
  GoodJob.preserve_job_records = true
262
270
  GoodJob.retry_on_unhandled_error = false
263
271
  GoodJob.on_thread_error = -> (exception) { Raven.capture_exception(exception) }
@@ -480,11 +488,11 @@ GoodJob can execute jobs "async" in the same process as the webserver (e.g. `bin
480
488
  config.active_job.queue_adapter = :good_job
481
489
 
482
490
  # To change the execution mode
483
- config.good_job.execution_mode = :async
491
+ config.good_job.execution_mode = :async_server
484
492
 
485
493
  # Or with more configuration
486
494
  config.good_job = {
487
- execution_mode: :async,
495
+ execution_mode: :async_server,
488
496
  max_threads: 4,
489
497
  poll_interval: 30
490
498
  }
@@ -493,7 +501,7 @@ GoodJob can execute jobs "async" in the same process as the webserver (e.g. `bin
493
501
  - Or, with environment variables:
494
502
 
495
503
  ```bash
496
- $ GOOD_JOB_EXECUTION_MODE=async GOOD_JOB_MAX_THREADS=4 GOOD_JOB_POLL_INTERVAL=30 bin/rails server
504
+ $ GOOD_JOB_EXECUTION_MODE=async_server GOOD_JOB_MAX_THREADS=4 GOOD_JOB_POLL_INTERVAL=30 bin/rails server
497
505
  ```
498
506
 
499
507
  Depending on your application configuration, you may need to take additional steps:
data/exe/good_job CHANGED
@@ -1,5 +1,6 @@
1
1
  #!/usr/bin/env ruby
2
2
  require 'good_job/cli'
3
- GOOD_JOB_WITHIN_CLI = true
4
- GOOD_JOB_LOG_TO_STDOUT = true
3
+
4
+ GoodJob::CLI.within_exe = true
5
+ GoodJob::CLI.log_to_stdout = true
5
6
  GoodJob::CLI.start(ARGV)
data/lib/active_job/queue_adapters/good_job_adapter.rb CHANGED
@@ -2,10 +2,6 @@ module ActiveJob # :nodoc:
2
2
  module QueueAdapters # :nodoc:
3
3
  # See {GoodJob::Adapter} for details.
4
4
  class GoodJobAdapter < GoodJob::Adapter
5
- def initialize(**options)
6
- configuration = GoodJob::Configuration.new(options, env: ENV)
7
- super(**options.merge(execution_mode: configuration.rails_execution_mode))
8
- end
9
5
  end
10
6
  end
11
7
  end
data/lib/good_job.rb CHANGED
@@ -17,6 +17,15 @@ require "good_job/railtie"
17
17
  #
18
18
  # +GoodJob+ is the top-level namespace and exposes configuration attributes.
19
19
  module GoodJob
20
+ # @!attribute [rw] active_record_parent_class
21
+ # @!scope class
22
+ # The ActiveRecord parent class inherited by +GoodJob::Job+ (default: +ActiveRecord::Base+).
23
+ # Use this when using multiple databases or other custom ActiveRecord configuration.
24
+ # @return [ActiveRecord::Base]
25
+ # @example Change the base class:
26
+ # GoodJob.active_record_parent_class = "CustomApplicationRecord"
27
+ mattr_accessor :active_record_parent_class, default: "ActiveRecord::Base"
28
+
20
29
  # @!attribute [rw] logger
21
30
  # @!scope class
22
31
  # The logger used by GoodJob (default: +Rails.logger+).
@@ -76,17 +85,33 @@ module GoodJob
76
85
  # When forking processes you should shut down these background threads before forking, and restart them after forking.
77
86
  # For example, you should use +shutdown+ and +restart+ when using async execution mode with Puma.
78
87
  # See the {file:README.md#executing-jobs-async--in-process} for more explanation and examples.
88
+ # @param timeout [nil, Numeric] Seconds to wait for actively executing jobs to finish
89
+ # * +nil+, the scheduler will trigger a shutdown but not wait for it to complete.
90
+ # * +-1+, the scheduler will wait until the shutdown is complete.
91
+ # * +0+, the scheduler will immediately shutdown and stop any active tasks.
92
+ # * +1..+, the scheduler will wait that many seconds before stopping any remaining active tasks.
79
93
  # @param wait [Boolean] whether to wait for shutdown
80
94
  # @return [void]
81
- def self.shutdown(wait: true)
82
- Notifier.instances.each { |notifier| notifier.shutdown(wait: wait) }
83
- Scheduler.instances.each { |scheduler| scheduler.shutdown(wait: wait) }
95
+ def self.shutdown(timeout: -1, wait: nil)
96
+ timeout = if wait.present?
97
+ ActiveSupport::Deprecation.warn(
98
+ "Using `GoodJob.shutdown` with `wait:` kwarg is deprecated; use `timeout:` kwarg instead e.g. GoodJob.shutdown(timeout: #{wait ? '-1' : 'nil'})"
99
+ )
100
+ wait ? -1 : nil
101
+ else
102
+ timeout
103
+ end
104
+
105
+ executables = Array(Notifier.instances) + Array(Poller.instances) + Array(Scheduler.instances)
106
+ _shutdown_all(executables, timeout: timeout)
84
107
  end
85
108
 
86
109
  # Tests whether jobs have stopped executing.
87
110
  # @return [Boolean] whether background threads are shut down
88
111
  def self.shutdown?
89
- Notifier.instances.all?(&:shutdown?) && Scheduler.instances.all?(&:shutdown?)
112
+ Notifier.instances.all?(&:shutdown?) &&
113
+ Poller.instances.all?(&:shutdown?) &&
114
+ Scheduler.instances.all?(&:shutdown?)
90
115
  end
91
116
 
92
117
  # Stops and restarts executing jobs.
@@ -95,9 +120,25 @@ module GoodJob
95
120
  # For example, you should use +shutdown+ and +restart+ when using async execution mode with Puma.
96
121
  # See the {file:README.md#executing-jobs-async--in-process} for more explanation and examples.
97
122
  # @return [void]
98
- def self.restart
99
- Notifier.instances.each(&:restart)
100
- Scheduler.instances.each(&:restart)
123
+ def self.restart(timeout: -1)
124
+ executables = Array(Notifier.instances) + Array(Poller.instances) + Array(Scheduler.instances)
125
+ _shutdown_all(executables, :restart, timeout: timeout)
126
+ end
127
+
128
+ # Sends +#shutdown+ or +#restart+ to executable objects ({GoodJob::Notifier}, {GoodJob::Poller}, {GoodJob::Scheduler})
129
+ # @param executables [Array<(Notifier, Poller, Scheduler)>] Objects to shut down.
130
+ # @param method_name [:symbol] Method to call, e.g. +:shutdown+ or +:restart+.
131
+ # @param timeout [nil,Numeric]
132
+ # @return [void]
133
+ def self._shutdown_all(executables, method_name = :shutdown, timeout: -1)
134
+ if timeout.positive?
135
+ executables.each { |executable| executable.send(method_name, timeout: nil) }
136
+
137
+ stop_at = Time.current + timeout
138
+ executables.each { |executable| executable.send(method_name, timeout: [stop_at - Time.current, 0].max) }
139
+ else
140
+ executables.each { |executable| executable.send(method_name, timeout: timeout) }
141
+ end
101
142
  end
102
143
 
103
144
  ActiveSupport.run_load_hooks(:good_job, self)
data/lib/good_job/adapter.rb CHANGED
@@ -4,13 +4,15 @@ module GoodJob
4
4
  #
5
5
  class Adapter
6
6
  # Valid execution modes.
7
- EXECUTION_MODES = [:async, :external, :inline].freeze
7
+ EXECUTION_MODES = [:async, :async_server, :external, :inline].freeze
8
8
 
9
9
  # @param execution_mode [nil, Symbol] specifies how and where jobs should be executed. You can also set this with the environment variable +GOOD_JOB_EXECUTION_MODE+.
10
10
  #
11
11
  # - +:inline+ executes jobs immediately in whatever process queued them (usually the web server process). This should only be used in test and development environments.
12
12
  # - +:external+ causes the adapter to enqueue jobs, but not execute them. When using this option (the default for production environments), you'll need to use the command-line tool to actually execute your jobs.
13
- # - +:async+ causes the adapter to execute you jobs in separate threads in whatever process queued them (usually the web process). This is akin to running the command-line tool's code inside your web server. It can be more economical for small workloads (you don't need a separate machine or environment for running your jobs), but if your web server is under heavy load or your jobs require a lot of resources, you should choose `:external` instead.
13
+ # - +:async_server+ executes jobs in separate threads within the Rails webserver process (`bundle exec rails server`). It can be more economical for small workloads because you don't need a separate machine or environment for running your jobs, but if your web server is under heavy load or your jobs require a lot of resources, you should choose +:external+ instead.
14
+ # When not in the Rails webserver, jobs will execute in +:external+ mode to ensure jobs are not executed within `rails console`, `rails db:migrate`, `rails assets:prepare`, etc.
15
+ # - +:async+ executes jobs in any Rails process.
14
16
  #
15
17
  # The default value depends on the Rails environment:
16
18
  #
@@ -38,7 +40,7 @@ module GoodJob
38
40
  DEPRECATION
39
41
  end
40
42
 
41
- configuration = GoodJob::Configuration.new(
43
+ @configuration = GoodJob::Configuration.new(
42
44
  {
43
45
  execution_mode: execution_mode,
44
46
  queues: queues,
@@ -46,14 +48,12 @@ module GoodJob
46
48
  poll_interval: poll_interval,
47
49
  }
48
50
  )
49
-
50
- @execution_mode = configuration.execution_mode
51
- raise ArgumentError, "execution_mode: must be one of #{EXECUTION_MODES.join(', ')}." unless EXECUTION_MODES.include?(@execution_mode)
51
+ @configuration.validate!
52
52
 
53
53
  if execute_async? # rubocop:disable Style/GuardClause
54
54
  @notifier = GoodJob::Notifier.new
55
- @poller = GoodJob::Poller.new(poll_interval: configuration.poll_interval)
56
- @scheduler = GoodJob::Scheduler.from_configuration(configuration, warm_cache_on_initialize: Rails.application.initialized?)
55
+ @poller = GoodJob::Poller.new(poll_interval: @configuration.poll_interval)
56
+ @scheduler = GoodJob::Scheduler.from_configuration(@configuration, warm_cache_on_initialize: Rails.application.initialized?)
57
57
  @notifier.recipients << [@scheduler, :create_thread]
58
58
  @poller.recipients << [@scheduler, :create_thread]
59
59
  end
@@ -96,29 +96,62 @@ module GoodJob
96
96
  good_job
97
97
  end
98
98
 
99
- # Gracefully stop processing jobs.
100
- # Waits for termination by default.
101
- # @param wait [Boolean] Whether to wait for shut down.
99
+ # Shut down the thread pool executors.
100
+ # @param timeout [nil, Numeric] Seconds to wait for active threads.
101
+ #
102
+ # * +nil+, the scheduler will trigger a shutdown but not wait for it to complete.
103
+ # * +-1+, the scheduler will wait until the shutdown is complete.
104
+ # * +0+, the scheduler will immediately shutdown and stop any threads.
105
+ # * A positive number will wait that many seconds before stopping any remaining active threads.
106
+ # @param wait [Boolean] Deprecated. Use +timeout:+ instead.
102
107
  # @return [void]
103
- def shutdown(wait: true)
104
- @notifier&.shutdown(wait: wait)
105
- @poller&.shutdown(wait: wait)
106
- @scheduler&.shutdown(wait: wait)
108
+ def shutdown(timeout: :default, wait: nil)
109
+ timeout = if wait.present?
110
+ ActiveSupport::Deprecation.warn(
111
+ "Using `GoodJob::Adapter.shutdown` with `wait:` kwarg is deprecated; use `timeout:` kwarg instead e.g. GoodJob::Adapter.shutdown(timeout: #{wait ? '-1' : 'nil'})"
112
+ )
113
+ wait ? -1 : nil
114
+ else
115
+ timeout
116
+ end
117
+
118
+ timeout = if timeout == :default
119
+ @configuration.shutdown_timeout
120
+ else
121
+ timeout
122
+ end
123
+
124
+ executables = [@notifier, @poller, @scheduler].compact
125
+ GoodJob._shutdown_all(executables, timeout: timeout)
107
126
  end
108
127
 
109
128
  # Whether in +:async+ execution mode.
110
129
  def execute_async?
111
- @execution_mode == :async
130
+ @configuration.execution_mode == :async ||
131
+ @configuration.execution_mode == :async_server && in_server_process?
112
132
  end
113
133
 
114
134
  # Whether in +:external+ execution mode.
115
135
  def execute_externally?
116
- @execution_mode == :external
136
+ @configuration.execution_mode == :external ||
137
+ @configuration.execution_mode == :async_server && !in_server_process?
117
138
  end
118
139
 
119
140
  # Whether in +:inline+ execution mode.
120
141
  def execute_inline?
121
- @execution_mode == :inline
142
+ @configuration.execution_mode == :inline
143
+ end
144
+
145
+ private
146
+
147
+ # Whether running in a web server process.
148
+ def in_server_process?
149
+ return @_in_server_process if defined? @_in_server_process
150
+
151
+ @_in_server_process = Rails.const_defined?('Server') ||
152
+ caller.grep(%r{config.ru}).any? || # EXAMPLE: config.ru:3:in `block in <main>' OR config.ru:3:in `new_from_string'
153
+ caller.grep(%{/rack/handler/}).any? || # EXAMPLE: iodine-0.7.44/lib/rack/handler/iodine.rb:13:in `start'
154
+ (Concurrent.on_jruby? && caller.grep(%r{jruby/rack/rails_booter}).any?) # EXAMPLE: uri:classloader:/jruby/rack/rails_booter.rb:83:in `load_environment'
122
155
  end
123
156
  end
124
157
  end