good_job 1.5.0 → 1.9.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/CHANGELOG.md +106 -9
- data/README.md +26 -15
- data/exe/good_job +3 -1
- data/lib/active_job/queue_adapters/good_job_adapter.rb +0 -4
- data/lib/good_job.rb +34 -7
- data/lib/good_job/adapter.rb +57 -22
- data/lib/good_job/cli.rb +38 -5
- data/lib/good_job/configuration.rb +86 -50
- data/lib/good_job/daemon.rb +59 -0
- data/lib/good_job/job.rb +25 -0
- data/lib/good_job/job_performer.rb +11 -0
- data/lib/good_job/lockable.rb +2 -2
- data/lib/good_job/multi_scheduler.rb +12 -7
- data/lib/good_job/notifier.rb +41 -32
- data/lib/good_job/poller.rb +32 -21
- data/lib/good_job/railtie.rb +4 -0
- data/lib/good_job/scheduler.rb +157 -52
- data/lib/good_job/version.rb +1 -1
- metadata +4 -17
checksums.yaml
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
SHA256:
|
|
3
|
-
metadata.gz:
|
|
4
|
-
data.tar.gz:
|
|
3
|
+
metadata.gz: 68ed9e19332edf1b6aa736b94980c1cc6eb08a3dbee67efad71852ed69220d23
|
|
4
|
+
data.tar.gz: bdf36f6d86f203de02dd647bd3a245564645e18331f97dda6dcca748876153d0
|
|
5
5
|
SHA512:
|
|
6
|
-
metadata.gz:
|
|
7
|
-
data.tar.gz:
|
|
6
|
+
metadata.gz: 6ecb2b42f43865aa9f9b1677438d727ff2c74ec003acd739fde3688d50dfc72e23bf61ab9cf8b12de519e31c56142a5ba79bbfa27bf09db74d829e95a1b11e30
|
|
7
|
+
data.tar.gz: f9c2038d1e4688cd83d23b0530071a447ee0b1692d7857a5e72b4814d8fb56496aca3aceb63b340542e518e8d0e9eeb29bf5a0ecf6d7f7da18825796cc6bee26
|
data/CHANGELOG.md
CHANGED
|
@@ -1,12 +1,110 @@
|
|
|
1
1
|
# Changelog
|
|
2
2
|
|
|
3
|
-
## [v1.
|
|
3
|
+
## [v1.9.0](https://github.com/bensheldon/good_job/tree/v1.9.0) (2021-04-16)
|
|
4
|
+
|
|
5
|
+
[Full Changelog](https://github.com/bensheldon/good_job/compare/v1.8.0...v1.9.0)
|
|
6
|
+
|
|
7
|
+
**Implemented enhancements:**
|
|
8
|
+
|
|
9
|
+
- Add `async_server` option to run async only in Rails web server process [\#230](https://github.com/bensheldon/good_job/pull/230) ([bensheldon](https://github.com/bensheldon))
|
|
10
|
+
- FreeBSD startup script [\#221](https://github.com/bensheldon/good_job/pull/221) ([lauer](https://github.com/lauer))
|
|
11
|
+
|
|
12
|
+
**Fixed bugs:**
|
|
13
|
+
|
|
14
|
+
- Fix instrumentation of GoodJob::Poller finished\_timer\_task event [\#233](https://github.com/bensheldon/good_job/pull/233) ([bensheldon](https://github.com/bensheldon))
|
|
15
|
+
|
|
16
|
+
**Closed issues:**
|
|
17
|
+
|
|
18
|
+
- Cannot run db:migrate when execution mode is :async [\#229](https://github.com/bensheldon/good_job/issues/229)
|
|
19
|
+
- How do you enqueue a job to be executed immediately outside of Rails \(eg. creating a new record of good\_jobs in Postgresql\)? [\#225](https://github.com/bensheldon/good_job/issues/225)
|
|
20
|
+
- Feature Ideas [\#220](https://github.com/bensheldon/good_job/issues/220)
|
|
21
|
+
- Goodjob startup script for FreeBSD [\#214](https://github.com/bensheldon/good_job/issues/214)
|
|
22
|
+
- Only start async mode executors when server is running [\#194](https://github.com/bensheldon/good_job/issues/194)
|
|
23
|
+
|
|
24
|
+
**Merged pull requests:**
|
|
25
|
+
|
|
26
|
+
- Move executable flags from constants to accessors on GoodJob::CLI [\#234](https://github.com/bensheldon/good_job/pull/234) ([bensheldon](https://github.com/bensheldon))
|
|
27
|
+
- Add custom Scheduler::TimerSet [\#232](https://github.com/bensheldon/good_job/pull/232) ([bensheldon](https://github.com/bensheldon))
|
|
28
|
+
- Fix assorted constant references in YARD documentation [\#231](https://github.com/bensheldon/good_job/pull/231) ([bensheldon](https://github.com/bensheldon))
|
|
29
|
+
- Update GH Test Matrix with latest JRuby 9.2.17.0 [\#228](https://github.com/bensheldon/good_job/pull/228) ([tedhexaflow](https://github.com/tedhexaflow))
|
|
30
|
+
- Update gem dependencies [\#227](https://github.com/bensheldon/good_job/pull/227) ([bensheldon](https://github.com/bensheldon))
|
|
31
|
+
- Remove leftover text from Readme [\#226](https://github.com/bensheldon/good_job/pull/226) ([weh](https://github.com/weh))
|
|
32
|
+
- Fix appraisal and bundler version CI conflicts [\#224](https://github.com/bensheldon/good_job/pull/224) ([bensheldon](https://github.com/bensheldon))
|
|
33
|
+
- Update GH Test Matrix with latest JRuby [\#223](https://github.com/bensheldon/good_job/pull/223) ([tedhexaflow](https://github.com/tedhexaflow))
|
|
34
|
+
|
|
35
|
+
## [v1.8.0](https://github.com/bensheldon/good_job/tree/v1.8.0) (2021-03-04)
|
|
36
|
+
|
|
37
|
+
[Full Changelog](https://github.com/bensheldon/good_job/compare/v1.7.1...v1.8.0)
|
|
38
|
+
|
|
39
|
+
**Implemented enhancements:**
|
|
40
|
+
|
|
41
|
+
- Wait then stop on shutdown [\#126](https://github.com/bensheldon/good_job/issues/126)
|
|
42
|
+
- Add shutdown-timeout option to configure the wait for jobs to gracefully finish before stopping them [\#213](https://github.com/bensheldon/good_job/pull/213) ([bensheldon](https://github.com/bensheldon))
|
|
43
|
+
|
|
44
|
+
**Fixed bugs:**
|
|
45
|
+
|
|
46
|
+
- Ensure Job\#serialized\_params are immutable [\#218](https://github.com/bensheldon/good_job/pull/218) ([bensheldon](https://github.com/bensheldon))
|
|
47
|
+
|
|
48
|
+
**Closed issues:**
|
|
49
|
+
|
|
50
|
+
- Run GoodJob on puma boot [\#91](https://github.com/bensheldon/good_job/issues/91)
|
|
51
|
+
- ActiveRecord::ConnectionNotEstablished when using async mode [\#89](https://github.com/bensheldon/good_job/issues/89)
|
|
52
|
+
|
|
53
|
+
**Merged pull requests:**
|
|
54
|
+
|
|
55
|
+
- Update bundler and Appraisals so Rails HEAD is locked to Ruby version \>= 2.7 [\#219](https://github.com/bensheldon/good_job/pull/219) ([bensheldon](https://github.com/bensheldon))
|
|
56
|
+
|
|
57
|
+
## [v1.7.1](https://github.com/bensheldon/good_job/tree/v1.7.1) (2021-01-27)
|
|
58
|
+
|
|
59
|
+
[Full Changelog](https://github.com/bensheldon/good_job/compare/v1.7.0...v1.7.1)
|
|
60
|
+
|
|
61
|
+
**Fixed bugs:**
|
|
62
|
+
|
|
63
|
+
- Scheduler should always push a new task on completion of previous task, regardless of available thread calculation [\#209](https://github.com/bensheldon/good_job/pull/209) ([bensheldon](https://github.com/bensheldon))
|
|
64
|
+
|
|
65
|
+
**Closed issues:**
|
|
66
|
+
|
|
67
|
+
- Unexpected behavior with max\_threads = 1 [\#208](https://github.com/bensheldon/good_job/issues/208)
|
|
68
|
+
|
|
69
|
+
**Merged pull requests:**
|
|
70
|
+
|
|
71
|
+
- Fix equality typo in development.rb of test\_app [\#207](https://github.com/bensheldon/good_job/pull/207) ([reczy](https://github.com/reczy))
|
|
72
|
+
|
|
73
|
+
## [v1.7.0](https://github.com/bensheldon/good_job/tree/v1.7.0) (2021-01-25)
|
|
74
|
+
|
|
75
|
+
[Full Changelog](https://github.com/bensheldon/good_job/compare/v1.6.0...v1.7.0)
|
|
76
|
+
|
|
77
|
+
**Implemented enhancements:**
|
|
78
|
+
|
|
79
|
+
- Cache scheduled jobs in memory so they can be executed without polling [\#205](https://github.com/bensheldon/good_job/pull/205) ([bensheldon](https://github.com/bensheldon))
|
|
80
|
+
|
|
81
|
+
## [v1.6.0](https://github.com/bensheldon/good_job/tree/v1.6.0) (2021-01-22)
|
|
82
|
+
|
|
83
|
+
[Full Changelog](https://github.com/bensheldon/good_job/compare/v1.5.0...v1.6.0)
|
|
84
|
+
|
|
85
|
+
**Implemented enhancements:**
|
|
86
|
+
|
|
87
|
+
- Running as a daemon [\#88](https://github.com/bensheldon/good_job/issues/88)
|
|
88
|
+
- Add daemonize option to CLI [\#202](https://github.com/bensheldon/good_job/pull/202) ([bensheldon](https://github.com/bensheldon))
|
|
89
|
+
|
|
90
|
+
**Closed issues:**
|
|
91
|
+
|
|
92
|
+
- Rails 6.1 & async - `queue_parser': undefined method `first' for "\*":String \(NoMethodError\) [\#195](https://github.com/bensheldon/good_job/issues/195)
|
|
93
|
+
|
|
94
|
+
**Merged pull requests:**
|
|
95
|
+
|
|
96
|
+
- Add scripts directory for benchmarking and dev tasks [\#204](https://github.com/bensheldon/good_job/pull/204) ([bensheldon](https://github.com/bensheldon))
|
|
97
|
+
- Fix YARD attr\_ declarations for documentation [\#203](https://github.com/bensheldon/good_job/pull/203) ([bensheldon](https://github.com/bensheldon))
|
|
98
|
+
- Remove Appraisal gemfile locks [\#201](https://github.com/bensheldon/good_job/pull/201) ([bensheldon](https://github.com/bensheldon))
|
|
99
|
+
|
|
100
|
+
## [v1.5.0](https://github.com/bensheldon/good_job/tree/v1.5.0) (2021-01-18)
|
|
4
101
|
|
|
5
102
|
[Full Changelog](https://github.com/bensheldon/good_job/compare/v1.4.1...v1.5.0)
|
|
6
103
|
|
|
7
104
|
**Implemented enhancements:**
|
|
8
105
|
|
|
9
106
|
- Create Web UI Dashboard [\#50](https://github.com/bensheldon/good_job/issues/50)
|
|
107
|
+
- Configure GoodJob via `Rails.application.config` instead of recommending `GoodJob::Adapter.new` [\#199](https://github.com/bensheldon/good_job/pull/199) ([bensheldon](https://github.com/bensheldon))
|
|
10
108
|
|
|
11
109
|
**Closed issues:**
|
|
12
110
|
|
|
@@ -15,7 +113,6 @@
|
|
|
15
113
|
**Merged pull requests:**
|
|
16
114
|
|
|
17
115
|
- Update bundler version to 2.2.5 [\#200](https://github.com/bensheldon/good_job/pull/200) ([bensheldon](https://github.com/bensheldon))
|
|
18
|
-
- Configure GoodJob via `Rails.application.config` instead of recommending `GoodJob::Adapter.new` [\#199](https://github.com/bensheldon/good_job/pull/199) ([bensheldon](https://github.com/bensheldon))
|
|
19
116
|
- Update GH Test Matrix with minimum & latest JRuby version [\#197](https://github.com/bensheldon/good_job/pull/197) ([tedhexaflow](https://github.com/tedhexaflow))
|
|
20
117
|
- Fix JRuby version number [\#193](https://github.com/bensheldon/good_job/pull/193) ([tedhexaflow](https://github.com/tedhexaflow))
|
|
21
118
|
|
|
@@ -41,7 +138,6 @@
|
|
|
41
138
|
|
|
42
139
|
**Implemented enhancements:**
|
|
43
140
|
|
|
44
|
-
- Format serialized params to ease reading [\#170](https://github.com/bensheldon/good_job/pull/170) ([morgoth](https://github.com/morgoth))
|
|
45
141
|
- Add JRuby support [\#167](https://github.com/bensheldon/good_job/pull/167) ([bensheldon](https://github.com/bensheldon))
|
|
46
142
|
|
|
47
143
|
## [v1.3.6](https://github.com/bensheldon/good_job/tree/v1.3.6) (2020-12-30)
|
|
@@ -117,6 +213,7 @@
|
|
|
117
213
|
**Implemented enhancements:**
|
|
118
214
|
|
|
119
215
|
- Extract polling from scheduler into Polling object [\#128](https://github.com/bensheldon/good_job/issues/128)
|
|
216
|
+
- Format serialized params to ease reading [\#170](https://github.com/bensheldon/good_job/pull/170) ([morgoth](https://github.com/morgoth))
|
|
120
217
|
|
|
121
218
|
**Fixed bugs:**
|
|
122
219
|
|
|
@@ -152,7 +249,7 @@
|
|
|
152
249
|
**Implemented enhancements:**
|
|
153
250
|
|
|
154
251
|
- Preserve only failed jobs [\#136](https://github.com/bensheldon/good_job/issues/136)
|
|
155
|
-
- Add `GoodJob.
|
|
252
|
+
- Add `GoodJob.preserve_job_records = :on_unhandled_error` option to only preserve jobs that errored [\#145](https://github.com/bensheldon/good_job/pull/145) ([morgoth](https://github.com/morgoth))
|
|
156
253
|
|
|
157
254
|
**Fixed bugs:**
|
|
158
255
|
|
|
@@ -185,7 +282,7 @@
|
|
|
185
282
|
- Update Gemspec to reflect that GoodJob is not compatible with Rails 5.1 [\#143](https://github.com/bensheldon/good_job/pull/143) ([bensheldon](https://github.com/bensheldon))
|
|
186
283
|
- Prevent jobs hanging [\#141](https://github.com/bensheldon/good_job/pull/141) ([morgoth](https://github.com/morgoth))
|
|
187
284
|
- Add explicit require\_paths to gemspec for engine [\#134](https://github.com/bensheldon/good_job/pull/134) ([bensheldon](https://github.com/bensheldon))
|
|
188
|
-
- Use `connection.
|
|
285
|
+
- Use `connection.quote_table_name` and add spacing for SQL concatenation [\#124](https://github.com/bensheldon/good_job/pull/124) ([bensheldon](https://github.com/bensheldon))
|
|
189
286
|
|
|
190
287
|
**Closed issues:**
|
|
191
288
|
|
|
@@ -211,7 +308,7 @@
|
|
|
211
308
|
|
|
212
309
|
**Implemented enhancements:**
|
|
213
310
|
|
|
214
|
-
- Add environment variable to mirror `
|
|
311
|
+
- Add environment variable to mirror `cleanup_preserved_jobs --before-seconds-ago=SECONDS` [\#110](https://github.com/bensheldon/good_job/issues/110)
|
|
215
312
|
- Allow env variable config for cleanups [\#114](https://github.com/bensheldon/good_job/pull/114) ([gadimbaylisahil](https://github.com/gadimbaylisahil))
|
|
216
313
|
|
|
217
314
|
**Fixed bugs:**
|
|
@@ -262,7 +359,7 @@
|
|
|
262
359
|
|
|
263
360
|
**Closed issues:**
|
|
264
361
|
|
|
265
|
-
- Add test for `rails g
|
|
362
|
+
- Add test for `rails g good_job:install` [\#57](https://github.com/bensheldon/good_job/issues/57)
|
|
266
363
|
|
|
267
364
|
**Merged pull requests:**
|
|
268
365
|
|
|
@@ -270,7 +367,7 @@
|
|
|
270
367
|
- Run CI tests on Ruby 2.5, 2.6, and 2.7 [\#101](https://github.com/bensheldon/good_job/pull/101) ([arku](https://github.com/arku))
|
|
271
368
|
- Fix Ruby 2.7 keyword arguments warning [\#98](https://github.com/bensheldon/good_job/pull/98) ([arku](https://github.com/arku))
|
|
272
369
|
- Remove executor/reloader for less interlocking [\#97](https://github.com/bensheldon/good_job/pull/97) ([sj26](https://github.com/sj26))
|
|
273
|
-
- Add test for `rails g
|
|
370
|
+
- Add test for `rails g good_job:install` [\#94](https://github.com/bensheldon/good_job/pull/94) ([arku](https://github.com/arku))
|
|
274
371
|
|
|
275
372
|
## [v1.2.1](https://github.com/bensheldon/good_job/tree/v1.2.1) (2020-08-21)
|
|
276
373
|
|
|
@@ -477,7 +574,7 @@
|
|
|
477
574
|
- Update Github Action Workflow for Backlog Project Board [\#35](https://github.com/bensheldon/good_job/pull/35) ([bensheldon](https://github.com/bensheldon))
|
|
478
575
|
- Add configuration options to good\_job executable [\#33](https://github.com/bensheldon/good_job/pull/33) ([bensheldon](https://github.com/bensheldon))
|
|
479
576
|
- Extract Job querying behavior out of Scheduler [\#31](https://github.com/bensheldon/good_job/pull/31) ([bensheldon](https://github.com/bensheldon))
|
|
480
|
-
- Allow configuration of Rails queue adapter with `:
|
|
577
|
+
- Allow configuration of Rails queue adapter with `:good_job` [\#28](https://github.com/bensheldon/good_job/pull/28) ([bensheldon](https://github.com/bensheldon))
|
|
481
578
|
|
|
482
579
|
## [v0.5.0](https://github.com/bensheldon/good_job/tree/v0.5.0) (2020-07-13)
|
|
483
580
|
|
data/README.md
CHANGED
|
@@ -36,7 +36,7 @@ For more of the story of GoodJob, read the [introductory blog post](https://isla
|
|
|
36
36
|
- [`good_job start`](#good_job-start)
|
|
37
37
|
- [`good_job cleanup_preserved_jobs`](#good_job-cleanup_preserved_jobs)
|
|
38
38
|
- [Configuration options](#configuration-options)
|
|
39
|
-
- [Global options](#global-options)
|
|
39
|
+
- [Global options](#global-options)
|
|
40
40
|
- [Dashboard](#dashboard)
|
|
41
41
|
- [Go deeper](#go-deeper)
|
|
42
42
|
- [Exceptions, retries, and reliability](#exceptions-retries-and-reliability)
|
|
@@ -119,10 +119,10 @@ For more of the story of GoodJob, read the [introductory blog post](https://isla
|
|
|
119
119
|
- GoodJob can also be configured to execute jobs within the web server process to save on resources. This is useful for low-workloads when economy is paramount.
|
|
120
120
|
|
|
121
121
|
```
|
|
122
|
-
$ GOOD_JOB_EXECUTION_MODE=
|
|
122
|
+
$ GOOD_JOB_EXECUTION_MODE=async_server rails server
|
|
123
123
|
```
|
|
124
124
|
|
|
125
|
-
Additional configuration is likely necessary, see the reference below for
|
|
125
|
+
Additional configuration is likely necessary, see the reference below for f configuration.
|
|
126
126
|
|
|
127
127
|
## Compatibility
|
|
128
128
|
|
|
@@ -149,9 +149,13 @@ Usage:
|
|
|
149
149
|
good_job start
|
|
150
150
|
|
|
151
151
|
Options:
|
|
152
|
-
[--max-threads=COUNT]
|
|
153
|
-
[--queues=QUEUE_LIST]
|
|
154
|
-
[--poll-interval=SECONDS]
|
|
152
|
+
[--max-threads=COUNT] # Maximum number of threads to use for working jobs. (env var: GOOD_JOB_MAX_THREADS, default: 5)
|
|
153
|
+
[--queues=QUEUE_LIST] # Queues to work from. (env var: GOOD_JOB_QUEUES, default: *)
|
|
154
|
+
[--poll-interval=SECONDS] # Interval between polls for available jobs in seconds (env var: GOOD_JOB_POLL_INTERVAL, default: 1)
|
|
155
|
+
[--max-cache=COUNT] # Maximum number of scheduled jobs to cache in memory (env var: GOOD_JOB_MAX_CACHE, default: 10000)
|
|
156
|
+
[--shutdown-timeout=SECONDS] # Number of seconds to wait for jobs to finish when shutting down before stopping the thread. (env var: GOOD_JOB_SHUTDOWN_TIMEOUT, default: -1 (forever))
|
|
157
|
+
[--daemonize] # Run as a background daemon (default: false)
|
|
158
|
+
[--pidfile=PIDFILE] # Path to write daemonized Process ID (env var: GOOD_JOB_PIDFILE, default: tmp/pids/good_job.pid)
|
|
155
159
|
|
|
156
160
|
Executes queued jobs.
|
|
157
161
|
|
|
@@ -159,6 +163,7 @@ All options can be configured with environment variables.
|
|
|
159
163
|
See option descriptions for the matching environment variable name.
|
|
160
164
|
|
|
161
165
|
== Configuring queues
|
|
166
|
+
|
|
162
167
|
Separate multiple queues with commas; exclude queues with a leading minus;
|
|
163
168
|
separate isolated execution pools with semicolons and threads with colons.
|
|
164
169
|
```
|
|
@@ -201,15 +206,18 @@ Additional configuration can be provided via `config.good_job.OPTION = ...` for
|
|
|
201
206
|
config.active_job.queue_adapter = :good_job
|
|
202
207
|
|
|
203
208
|
# Configure options individually...
|
|
204
|
-
config.good_job.execution_mode = :
|
|
209
|
+
config.good_job.execution_mode = :async_server
|
|
205
210
|
config.good_job.max_threads = 5
|
|
206
211
|
config.good_job.poll_interval = 30 # seconds
|
|
212
|
+
config.good_job.shutdown_timeout = 25 # seconds
|
|
213
|
+
|
|
207
214
|
|
|
208
215
|
# ...or all at once.
|
|
209
216
|
config.good_job = {
|
|
210
|
-
execution_mode: :
|
|
217
|
+
execution_mode: :async_server,
|
|
211
218
|
max_threads: 5,
|
|
212
219
|
poll_interval: 30,
|
|
220
|
+
shutdown_timeout: 25,
|
|
213
221
|
}
|
|
214
222
|
```
|
|
215
223
|
|
|
@@ -218,10 +226,13 @@ Available configuration options are:
|
|
|
218
226
|
- `execution_mode` (symbol) specifies how and where jobs should be executed. You can also set this with the environment variable `GOOD_JOB_EXECUTION_MODE`. It can be any one of:
|
|
219
227
|
- `:inline` executes jobs immediately in whatever process queued them (usually the web server process). This should only be used in test and development environments.
|
|
220
228
|
- `:external` causes the adapter to enqueue jobs, but not execute them. When using this option (the default for production environments), you’ll need to use the command-line tool to actually execute your jobs.
|
|
221
|
-
- `:
|
|
222
|
-
- `
|
|
223
|
-
- `
|
|
224
|
-
- `
|
|
229
|
+
- `:async_server` executes jobs in separate threads within the Rails webserver process (`bundle exec rails server`). It can be more economical for small workloads because you don’t need a separate machine or environment for running your jobs, but if your web server is under heavy load or your jobs require a lot of resources, you should choose `:external` instead. When not in the Rails webserver, jobs will execute in `:external` mode to ensure jobs are not executed within `rails console`, `rails db:migrate`, `rails assets:prepare`, etc.
|
|
230
|
+
- `:async` executes jobs in separate threads in _any_ Rails process.
|
|
231
|
+
- `max_threads` (integer) sets the maximum number of threads to use when `execution_mode` is set to `:async` or `:async_server`. You can also set this with the environment variable `GOOD_JOB_MAX_THREADS`.
|
|
232
|
+
- `queues` (string) determines which queues to execute jobs from when `execution_mode` is set to `:async` or `:async_server`. See the description of `good_job start` for more details on the format of this string. You can also set this with the environment variable `GOOD_JOB_QUEUES`.
|
|
233
|
+
- `poll_interval` (integer) sets the number of seconds between polls for jobs when `execution_mode` is set to `:async` or `:async_server`. You can also set this with the environment variable `GOOD_JOB_POLL_INTERVAL`.
|
|
234
|
+
- `max_cache` (integer) sets the maximum number of scheduled jobs that will be stored in memory to reduce execution latency when also polling for scheduled jobs. Caching 10,000 scheduled jobs uses approximately 20MB of memory. You can also set this with the environment variable `GOOD_JOB_MAX_CACHE`.
|
|
235
|
+
- `shutdown_timeout` (float) number of seconds to wait for jobs to finish when shutting down before stopping the thread. Defaults to forever: `-1`. You can also set this with the environment variable `GOOD_JOB_SHUTDOWN_TIMEOUT`.
|
|
225
236
|
|
|
226
237
|
By default, GoodJob configures the following execution modes per environment:
|
|
227
238
|
|
|
@@ -475,11 +486,11 @@ GoodJob can execute jobs "async" in the same process as the webserver (e.g. `bin
|
|
|
475
486
|
config.active_job.queue_adapter = :good_job
|
|
476
487
|
|
|
477
488
|
# To change the execution mode
|
|
478
|
-
config.good_job.execution_mode = :
|
|
489
|
+
config.good_job.execution_mode = :async_server
|
|
479
490
|
|
|
480
491
|
# Or with more configuration
|
|
481
492
|
config.good_job = {
|
|
482
|
-
execution_mode: :
|
|
493
|
+
execution_mode: :async_server,
|
|
483
494
|
max_threads: 4,
|
|
484
495
|
poll_interval: 30
|
|
485
496
|
}
|
|
@@ -488,7 +499,7 @@ GoodJob can execute jobs "async" in the same process as the webserver (e.g. `bin
|
|
|
488
499
|
- Or, with environment variables:
|
|
489
500
|
|
|
490
501
|
```bash
|
|
491
|
-
$ GOOD_JOB_EXECUTION_MODE=
|
|
502
|
+
$ GOOD_JOB_EXECUTION_MODE=async_server GOOD_JOB_MAX_THREADS=4 GOOD_JOB_POLL_INTERVAL=30 bin/rails server
|
|
492
503
|
```
|
|
493
504
|
|
|
494
505
|
Depending on your application configuration, you may need to take additional steps:
|
data/exe/good_job
CHANGED
|
@@ -2,10 +2,6 @@ module ActiveJob # :nodoc:
|
|
|
2
2
|
module QueueAdapters # :nodoc:
|
|
3
3
|
# See {GoodJob::Adapter} for details.
|
|
4
4
|
class GoodJobAdapter < GoodJob::Adapter
|
|
5
|
-
def initialize(**options)
|
|
6
|
-
configuration = GoodJob::Configuration.new(options, env: ENV)
|
|
7
|
-
super(**options.merge(execution_mode: configuration.rails_execution_mode))
|
|
8
|
-
end
|
|
9
5
|
end
|
|
10
6
|
end
|
|
11
7
|
end
|
data/lib/good_job.rb
CHANGED
|
@@ -78,15 +78,26 @@ module GoodJob
|
|
|
78
78
|
# See the {file:README.md#executing-jobs-async--in-process} for more explanation and examples.
|
|
79
79
|
# @param wait [Boolean] whether to wait for shutdown
|
|
80
80
|
# @return [void]
|
|
81
|
-
def self.shutdown(wait:
|
|
82
|
-
|
|
83
|
-
|
|
81
|
+
def self.shutdown(timeout: -1, wait: nil)
|
|
82
|
+
timeout = if wait.present?
|
|
83
|
+
ActiveSupport::Deprecation.warn(
|
|
84
|
+
"Using `GoodJob.shutdown` with `wait:` kwarg is deprecated; use `timeout:` kwarg instead e.g. GoodJob.shutdown(timeout: #{wait ? '-1' : 'nil'})"
|
|
85
|
+
)
|
|
86
|
+
wait ? -1 : nil
|
|
87
|
+
else
|
|
88
|
+
timeout
|
|
89
|
+
end
|
|
90
|
+
|
|
91
|
+
executables = Array(Notifier.instances) + Array(Poller.instances) + Array(Scheduler.instances)
|
|
92
|
+
_shutdown_all(executables, timeout: timeout)
|
|
84
93
|
end
|
|
85
94
|
|
|
86
95
|
# Tests whether jobs have stopped executing.
|
|
87
96
|
# @return [Boolean] whether background threads are shut down
|
|
88
97
|
def self.shutdown?
|
|
89
|
-
Notifier.instances.all?(&:shutdown?) &&
|
|
98
|
+
Notifier.instances.all?(&:shutdown?) &&
|
|
99
|
+
Poller.instances.all?(&:shutdown?) &&
|
|
100
|
+
Scheduler.instances.all?(&:shutdown?)
|
|
90
101
|
end
|
|
91
102
|
|
|
92
103
|
# Stops and restarts executing jobs.
|
|
@@ -95,9 +106,25 @@ module GoodJob
|
|
|
95
106
|
# For example, you should use +shutdown+ and +restart+ when using async execution mode with Puma.
|
|
96
107
|
# See the {file:README.md#executing-jobs-async--in-process} for more explanation and examples.
|
|
97
108
|
# @return [void]
|
|
98
|
-
def self.restart
|
|
99
|
-
Notifier.instances.
|
|
100
|
-
|
|
109
|
+
def self.restart(timeout: -1)
|
|
110
|
+
executables = Array(Notifier.instances) + Array(Poller.instances) + Array(Scheduler.instances)
|
|
111
|
+
_shutdown_all(executables, :restart, timeout: timeout)
|
|
112
|
+
end
|
|
113
|
+
|
|
114
|
+
# Sends +#shutdown+ or +#restart+ to executable objects ({GoodJob::Notifier}, {GoodJob::Poller}, {GoodJob::Scheduler})
|
|
115
|
+
# @param executables [Array<(Notifier, Poller, Scheduler)>] Objects to shut down.
|
|
116
|
+
# @param method_name [:symbol] Method to call, e.g. +:shutdown+ or +:restart+.
|
|
117
|
+
# @param timeout [nil,Numeric]
|
|
118
|
+
# @return [void]
|
|
119
|
+
def self._shutdown_all(executables, method_name = :shutdown, timeout: -1)
|
|
120
|
+
if timeout.positive?
|
|
121
|
+
executables.each { |executable| executable.send(method_name, timeout: nil) }
|
|
122
|
+
|
|
123
|
+
stop_at = Time.current + timeout
|
|
124
|
+
executables.each { |executable| executable.send(method_name, timeout: [stop_at - Time.current, 0].max) }
|
|
125
|
+
else
|
|
126
|
+
executables.each { |executable| executable.send(method_name, timeout: timeout) }
|
|
127
|
+
end
|
|
101
128
|
end
|
|
102
129
|
|
|
103
130
|
ActiveSupport.run_load_hooks(:good_job, self)
|
data/lib/good_job/adapter.rb
CHANGED
|
@@ -4,13 +4,15 @@ module GoodJob
|
|
|
4
4
|
#
|
|
5
5
|
class Adapter
|
|
6
6
|
# Valid execution modes.
|
|
7
|
-
EXECUTION_MODES = [:async, :external, :inline].freeze
|
|
7
|
+
EXECUTION_MODES = [:async, :async_server, :external, :inline].freeze
|
|
8
8
|
|
|
9
9
|
# @param execution_mode [nil, Symbol] specifies how and where jobs should be executed. You can also set this with the environment variable +GOOD_JOB_EXECUTION_MODE+.
|
|
10
10
|
#
|
|
11
11
|
# - +:inline+ executes jobs immediately in whatever process queued them (usually the web server process). This should only be used in test and development environments.
|
|
12
12
|
# - +:external+ causes the adapter to enqueue jobs, but not execute them. When using this option (the default for production environments), you'll need to use the command-line tool to actually execute your jobs.
|
|
13
|
-
# - +:
|
|
13
|
+
# - +:async_server+ executes jobs in separate threads within the Rails webserver process (`bundle exec rails server`). It can be more economical for small workloads because you don't need a separate machine or environment for running your jobs, but if your web server is under heavy load or your jobs require a lot of resources, you should choose +:external+ instead.
|
|
14
|
+
# When not in the Rails webserver, jobs will execute in +:external+ mode to ensure jobs are not executed within `rails console`, `rails db:migrate`, `rails assets:prepare`, etc.
|
|
15
|
+
# - +:async+ executes jobs in any Rails process.
|
|
14
16
|
#
|
|
15
17
|
# The default value depends on the Rails environment:
|
|
16
18
|
#
|
|
@@ -38,7 +40,7 @@ module GoodJob
|
|
|
38
40
|
DEPRECATION
|
|
39
41
|
end
|
|
40
42
|
|
|
41
|
-
configuration = GoodJob::Configuration.new(
|
|
43
|
+
@configuration = GoodJob::Configuration.new(
|
|
42
44
|
{
|
|
43
45
|
execution_mode: execution_mode,
|
|
44
46
|
queues: queues,
|
|
@@ -46,14 +48,12 @@ module GoodJob
|
|
|
46
48
|
poll_interval: poll_interval,
|
|
47
49
|
}
|
|
48
50
|
)
|
|
51
|
+
@configuration.validate!
|
|
49
52
|
|
|
50
|
-
|
|
51
|
-
raise ArgumentError, "execution_mode: must be one of #{EXECUTION_MODES.join(', ')}." unless EXECUTION_MODES.include?(@execution_mode)
|
|
52
|
-
|
|
53
|
-
if @execution_mode == :async # rubocop:disable Style/GuardClause
|
|
53
|
+
if execute_async? # rubocop:disable Style/GuardClause
|
|
54
54
|
@notifier = GoodJob::Notifier.new
|
|
55
|
-
@poller = GoodJob::Poller.new(poll_interval: configuration.poll_interval)
|
|
56
|
-
@scheduler = GoodJob::Scheduler.from_configuration(configuration)
|
|
55
|
+
@poller = GoodJob::Poller.new(poll_interval: @configuration.poll_interval)
|
|
56
|
+
@scheduler = GoodJob::Scheduler.from_configuration(@configuration, warm_cache_on_initialize: Rails.application.initialized?)
|
|
57
57
|
@notifier.recipients << [@scheduler, :create_thread]
|
|
58
58
|
@poller.recipients << [@scheduler, :create_thread]
|
|
59
59
|
end
|
|
@@ -85,37 +85,72 @@ module GoodJob
|
|
|
85
85
|
ensure
|
|
86
86
|
good_job.advisory_unlock
|
|
87
87
|
end
|
|
88
|
-
|
|
88
|
+
else
|
|
89
|
+
job_state = { queue_name: good_job.queue_name }
|
|
90
|
+
job_state[:scheduled_at] = good_job.scheduled_at if good_job.scheduled_at
|
|
89
91
|
|
|
90
|
-
|
|
91
|
-
|
|
92
|
+
executed_locally = execute_async? && @scheduler.create_thread(job_state)
|
|
93
|
+
Notifier.notify(job_state) unless executed_locally
|
|
94
|
+
end
|
|
92
95
|
|
|
93
96
|
good_job
|
|
94
97
|
end
|
|
95
98
|
|
|
96
|
-
#
|
|
97
|
-
#
|
|
98
|
-
#
|
|
99
|
+
# Shut down the thread pool executors.
|
|
100
|
+
# @param timeout [nil, Numeric] Seconds to wait for active threads.
|
|
101
|
+
#
|
|
102
|
+
# * +nil+, the scheduler will trigger a shutdown but not wait for it to complete.
|
|
103
|
+
# * +-1+, the scheduler will wait until the shutdown is complete.
|
|
104
|
+
# * +0+, the scheduler will immediately shutdown and stop any threads.
|
|
105
|
+
# * A positive number will wait that many seconds before stopping any remaining active threads.
|
|
106
|
+
# @param wait [Boolean] Deprecated. Use +timeout:+ instead.
|
|
99
107
|
# @return [void]
|
|
100
|
-
def shutdown(wait:
|
|
101
|
-
|
|
102
|
-
|
|
103
|
-
|
|
108
|
+
def shutdown(timeout: :default, wait: nil)
|
|
109
|
+
timeout = if wait.present?
|
|
110
|
+
ActiveSupport::Deprecation.warn(
|
|
111
|
+
"Using `GoodJob::Adapter.shutdown` with `wait:` kwarg is deprecated; use `timeout:` kwarg instead e.g. GoodJob::Adapter.shutdown(timeout: #{wait ? '-1' : 'nil'})"
|
|
112
|
+
)
|
|
113
|
+
wait ? -1 : nil
|
|
114
|
+
else
|
|
115
|
+
timeout
|
|
116
|
+
end
|
|
117
|
+
|
|
118
|
+
timeout = if timeout == :default
|
|
119
|
+
@configuration.shutdown_timeout
|
|
120
|
+
else
|
|
121
|
+
timeout
|
|
122
|
+
end
|
|
123
|
+
|
|
124
|
+
executables = [@notifier, @poller, @scheduler].compact
|
|
125
|
+
GoodJob._shutdown_all(executables, timeout: timeout)
|
|
104
126
|
end
|
|
105
127
|
|
|
106
128
|
# Whether in +:async+ execution mode.
|
|
107
129
|
def execute_async?
|
|
108
|
-
@execution_mode == :async
|
|
130
|
+
@configuration.execution_mode == :async ||
|
|
131
|
+
@configuration.execution_mode == :async_server && in_server_process?
|
|
109
132
|
end
|
|
110
133
|
|
|
111
134
|
# Whether in +:external+ execution mode.
|
|
112
135
|
def execute_externally?
|
|
113
|
-
@execution_mode == :external
|
|
136
|
+
@configuration.execution_mode == :external ||
|
|
137
|
+
@configuration.execution_mode == :async_server && !in_server_process?
|
|
114
138
|
end
|
|
115
139
|
|
|
116
140
|
# Whether in +:inline+ execution mode.
|
|
117
141
|
def execute_inline?
|
|
118
|
-
@execution_mode == :inline
|
|
142
|
+
@configuration.execution_mode == :inline
|
|
143
|
+
end
|
|
144
|
+
|
|
145
|
+
private
|
|
146
|
+
|
|
147
|
+
# Whether running in a web server process.
|
|
148
|
+
def in_server_process?
|
|
149
|
+
return @_in_server_process if defined? @_in_server_process
|
|
150
|
+
|
|
151
|
+
@_in_server_process = Rails.const_defined?('Server') ||
|
|
152
|
+
caller.grep(%r{config.ru}).any? || # EXAMPLE: config.ru:3:in `block in <main>' OR config.ru:3:in `new_from_string'
|
|
153
|
+
(Concurrent.on_jruby? && caller.grep(%r{jruby/rack/rails_booter}).any?) # EXAMPLE: uri:classloader:/jruby/rack/rails_booter.rb:83:in `load_environment'
|
|
119
154
|
end
|
|
120
155
|
end
|
|
121
156
|
end
|