good_job 1.2.5 → 1.3.3
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/CHANGELOG.md +93 -9
- data/README.md +64 -15
- data/engine/app/controllers/good_job/dashboards_controller.rb +52 -3
- data/engine/app/views/good_job/dashboards/index.html.erb +26 -3
- data/engine/app/views/layouts/good_job/base.html.erb +23 -12
- data/engine/app/views/shared/_jobs_table.erb +3 -3
- data/lib/good_job.rb +19 -2
- data/lib/good_job/adapter.rb +3 -0
- data/lib/good_job/cli.rb +5 -2
- data/lib/good_job/configuration.rb +10 -3
- data/lib/good_job/job.rb +16 -22
- data/lib/good_job/lockable.rb +7 -7
- data/lib/good_job/log_subscriber.rb +12 -14
- data/lib/good_job/multi_scheduler.rb +14 -5
- data/lib/good_job/notifier.rb +3 -3
- data/lib/good_job/poller.rb +94 -0
- data/lib/good_job/scheduler.rb +30 -70
- data/lib/good_job/version.rb +1 -1
- metadata +18 -3
checksums.yaml
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
---
|
2
2
|
SHA256:
|
3
|
-
metadata.gz:
|
4
|
-
data.tar.gz:
|
3
|
+
metadata.gz: d48414f6b03482087018157412c87afb3fdb70929a292294f7db2e14309c31fd
|
4
|
+
data.tar.gz: dd221c88385350fff14b593a6504bc98adeae44893d523c90949332a81eb6fd9
|
5
5
|
SHA512:
|
6
|
-
metadata.gz:
|
7
|
-
data.tar.gz:
|
6
|
+
metadata.gz: 77ab00b0cd0772641a402205856d5c03c675dc2edc769e5fed8909636fc2c4a69b033da98a7d0ae09a7ac1fc3f40dd5dc4ff637b3d13e4c290acb9a40bc51ead
|
7
|
+
data.tar.gz: 1777e663267f7e626d7f5ebf5b61b156e83b7bcf9234addddabeb10ef1054c05f8602ca0111c1abe4de9fb53b32aafac4df5d88e1c7c0147e0f1a7cbdc57219d
|
data/CHANGELOG.md
CHANGED
@@ -1,5 +1,86 @@
|
|
1
1
|
# Changelog
|
2
2
|
|
3
|
+
## [v1.3.3](https://github.com/bensheldon/good_job/tree/v1.3.3) (2020-12-01)
|
4
|
+
|
5
|
+
[Full Changelog](https://github.com/bensheldon/good_job/compare/v1.3.2...v1.3.3)
|
6
|
+
|
7
|
+
**Merged pull requests:**
|
8
|
+
|
9
|
+
- UI: Admin UI with filters and space efficient layout [\#173](https://github.com/bensheldon/good_job/pull/173) ([zealot128](https://github.com/zealot128))
|
10
|
+
|
11
|
+
## [v1.3.2](https://github.com/bensheldon/good_job/tree/v1.3.2) (2020-11-12)
|
12
|
+
|
13
|
+
[Full Changelog](https://github.com/bensheldon/good_job/compare/v1.3.1...v1.3.2)
|
14
|
+
|
15
|
+
**Fixed bugs:**
|
16
|
+
|
17
|
+
- \(bug\) MultiScheduler polling bug [\#171](https://github.com/bensheldon/good_job/issues/171)
|
18
|
+
- MultiScheduler should delegate to all schedulers when state is nil [\#172](https://github.com/bensheldon/good_job/pull/172) ([bensheldon](https://github.com/bensheldon))
|
19
|
+
|
20
|
+
## [v1.3.1](https://github.com/bensheldon/good_job/tree/v1.3.1) (2020-11-01)
|
21
|
+
|
22
|
+
[Full Changelog](https://github.com/bensheldon/good_job/compare/v1.3.0...v1.3.1)
|
23
|
+
|
24
|
+
**Implemented enhancements:**
|
25
|
+
|
26
|
+
- Extract polling from scheduler into Polling object [\#128](https://github.com/bensheldon/good_job/issues/128)
|
27
|
+
- Format serialized params to ease reading [\#170](https://github.com/bensheldon/good_job/pull/170) ([morgoth](https://github.com/morgoth))
|
28
|
+
|
29
|
+
**Fixed bugs:**
|
30
|
+
|
31
|
+
- Don't disconnect a nil activerecord connection [\#161](https://github.com/bensheldon/good_job/pull/161) ([bensheldon](https://github.com/bensheldon))
|
32
|
+
|
33
|
+
**Closed issues:**
|
34
|
+
|
35
|
+
- Propose addition of GoodJob to queue-shootout benchmarks [\#40](https://github.com/bensheldon/good_job/issues/40)
|
36
|
+
|
37
|
+
**Merged pull requests:**
|
38
|
+
|
39
|
+
- Ensure Rails is a development dependency [\#169](https://github.com/bensheldon/good_job/pull/169) ([bensheldon](https://github.com/bensheldon))
|
40
|
+
- Fix Ruby 2.7 GH action by setting default bundler explicitly [\#166](https://github.com/bensheldon/good_job/pull/166) ([bensheldon](https://github.com/bensheldon))
|
41
|
+
- Cache ruby version explicitly in Github Action [\#165](https://github.com/bensheldon/good_job/pull/165) ([bensheldon](https://github.com/bensheldon))
|
42
|
+
- Update development dependencies, rubocop [\#164](https://github.com/bensheldon/good_job/pull/164) ([bensheldon](https://github.com/bensheldon))
|
43
|
+
- Fix intended constant hierarchy of GoodJob::Scheduler::ThreadPoolExecutor [\#158](https://github.com/bensheldon/good_job/pull/158) ([bensheldon](https://github.com/bensheldon))
|
44
|
+
- Add bin/test\_app executable for Rails debugging [\#157](https://github.com/bensheldon/good_job/pull/157) ([bensheldon](https://github.com/bensheldon))
|
45
|
+
- Extract Scheduler polling behavior to its own object [\#152](https://github.com/bensheldon/good_job/pull/152) ([bensheldon](https://github.com/bensheldon))
|
46
|
+
|
47
|
+
## [v1.3.0](https://github.com/bensheldon/good_job/tree/v1.3.0) (2020-10-03)
|
48
|
+
|
49
|
+
[Full Changelog](https://github.com/bensheldon/good_job/compare/v1.2.6...v1.3.0)
|
50
|
+
|
51
|
+
**Implemented enhancements:**
|
52
|
+
|
53
|
+
- Lengthen default poll interval from 1 to 5 seconds [\#156](https://github.com/bensheldon/good_job/pull/156) ([bensheldon](https://github.com/bensheldon))
|
54
|
+
|
55
|
+
**Merged pull requests:**
|
56
|
+
|
57
|
+
- Rename reperform\_jobs\_on\_standard\_error to retry\_on\_unhandled\_error [\#154](https://github.com/bensheldon/good_job/pull/154) ([morgoth](https://github.com/morgoth))
|
58
|
+
|
59
|
+
## [v1.2.6](https://github.com/bensheldon/good_job/tree/v1.2.6) (2020-09-29)
|
60
|
+
|
61
|
+
[Full Changelog](https://github.com/bensheldon/good_job/compare/v1.2.5...v1.2.6)
|
62
|
+
|
63
|
+
**Implemented enhancements:**
|
64
|
+
|
65
|
+
- Preserve only failed jobs [\#136](https://github.com/bensheldon/good_job/issues/136)
|
66
|
+
- Add `GoodJob.preserve\_job\_records = :on\_unhandled\_error` option to only preserve jobs that errored [\#145](https://github.com/bensheldon/good_job/pull/145) ([morgoth](https://github.com/morgoth))
|
67
|
+
|
68
|
+
**Fixed bugs:**
|
69
|
+
|
70
|
+
- Fix LogSubscriber notifications for finished\_timer\_task and finished\_job\_task [\#148](https://github.com/bensheldon/good_job/pull/148) ([bensheldon](https://github.com/bensheldon))
|
71
|
+
|
72
|
+
**Closed issues:**
|
73
|
+
|
74
|
+
- run-once guarantee? [\#151](https://github.com/bensheldon/good_job/issues/151)
|
75
|
+
|
76
|
+
**Merged pull requests:**
|
77
|
+
|
78
|
+
- Add info how to setup basic auth for engine [\#153](https://github.com/bensheldon/good_job/pull/153) ([morgoth](https://github.com/morgoth))
|
79
|
+
- Add documentation for Dashboard Rails::Engine [\#149](https://github.com/bensheldon/good_job/pull/149) ([bensheldon](https://github.com/bensheldon))
|
80
|
+
- Style cleanup to Job error handling [\#147](https://github.com/bensheldon/good_job/pull/147) ([bensheldon](https://github.com/bensheldon))
|
81
|
+
- Replace gerund titles in Readme [\#146](https://github.com/bensheldon/good_job/pull/146) ([bensheldon](https://github.com/bensheldon))
|
82
|
+
- Only allow Scheduler to be initialized with max\_threads and poll\_interval; remove full access to pool and timer\_task options [\#137](https://github.com/bensheldon/good_job/pull/137) ([bensheldon](https://github.com/bensheldon))
|
83
|
+
|
3
84
|
## [v1.2.5](https://github.com/bensheldon/good_job/tree/v1.2.5) (2020-09-17)
|
4
85
|
|
5
86
|
[Full Changelog](https://github.com/bensheldon/good_job/compare/v1.2.4...v1.2.5)
|
@@ -7,10 +88,15 @@
|
|
7
88
|
**Implemented enhancements:**
|
8
89
|
|
9
90
|
- Use Zeitwerk for auto-loading [\#87](https://github.com/bensheldon/good_job/issues/87)
|
91
|
+
- Spike on data dashboard; pull in full Bootstrap CSS and JS [\#131](https://github.com/bensheldon/good_job/pull/131) ([bensheldon](https://github.com/bensheldon))
|
10
92
|
|
11
93
|
**Fixed bugs:**
|
12
94
|
|
13
95
|
- `poll-interval=-1` does not disable polling as intended [\#133](https://github.com/bensheldon/good_job/issues/133)
|
96
|
+
- Update Gemspec to reflect that GoodJob is not compatible with Rails 5.1 [\#143](https://github.com/bensheldon/good_job/pull/143) ([bensheldon](https://github.com/bensheldon))
|
97
|
+
- Prevent jobs hanging [\#141](https://github.com/bensheldon/good_job/pull/141) ([morgoth](https://github.com/morgoth))
|
98
|
+
- Add explicit require\_paths to gemspec for engine [\#134](https://github.com/bensheldon/good_job/pull/134) ([bensheldon](https://github.com/bensheldon))
|
99
|
+
- Use `connection.quote\_table\_name` and add spacing for SQL concatenation [\#124](https://github.com/bensheldon/good_job/pull/124) ([bensheldon](https://github.com/bensheldon))
|
14
100
|
|
15
101
|
**Closed issues:**
|
16
102
|
|
@@ -21,17 +107,12 @@
|
|
21
107
|
**Merged pull requests:**
|
22
108
|
|
23
109
|
- Test GoodJob against Rails HEAD [\#144](https://github.com/bensheldon/good_job/pull/144) ([bensheldon](https://github.com/bensheldon))
|
24
|
-
- Update Gemspec to reflect that GoodJob is not compatible with Rails 5.1 [\#143](https://github.com/bensheldon/good_job/pull/143) ([bensheldon](https://github.com/bensheldon))
|
25
110
|
- Drop Ruby 2.4 support [\#142](https://github.com/bensheldon/good_job/pull/142) ([morgoth](https://github.com/morgoth))
|
26
|
-
- Prevent jobs hanging [\#141](https://github.com/bensheldon/good_job/pull/141) ([morgoth](https://github.com/morgoth))
|
27
111
|
- Remove arguments from perform method [\#140](https://github.com/bensheldon/good_job/pull/140) ([morgoth](https://github.com/morgoth))
|
28
112
|
- Extract "execute" method to reduce "perform" method complexity [\#138](https://github.com/bensheldon/good_job/pull/138) ([morgoth](https://github.com/morgoth))
|
29
113
|
- Correct example on how to configure multiple queues by command line. [\#135](https://github.com/bensheldon/good_job/pull/135) ([morgoth](https://github.com/morgoth))
|
30
|
-
- Add explicit require\_paths to gemspec for engine [\#134](https://github.com/bensheldon/good_job/pull/134) ([bensheldon](https://github.com/bensheldon))
|
31
|
-
- Spike on data dashboard; pull in full Bootstrap CSS and JS [\#131](https://github.com/bensheldon/good_job/pull/131) ([bensheldon](https://github.com/bensheldon))
|
32
114
|
- Update ActionMailer Job class, to match the default [\#130](https://github.com/bensheldon/good_job/pull/130) ([morgoth](https://github.com/morgoth))
|
33
115
|
- Add initial Engine scaffold [\#125](https://github.com/bensheldon/good_job/pull/125) ([bensheldon](https://github.com/bensheldon))
|
34
|
-
- Use `connection.quote\_table\_name` and add spacing for SQL concatenation [\#124](https://github.com/bensheldon/good_job/pull/124) ([bensheldon](https://github.com/bensheldon))
|
35
116
|
- Zeitwerk Loader Implementation [\#123](https://github.com/bensheldon/good_job/pull/123) ([gadimbaylisahil](https://github.com/gadimbaylisahil))
|
36
117
|
- Update code-level documentation [\#111](https://github.com/bensheldon/good_job/pull/111) ([bensheldon](https://github.com/bensheldon))
|
37
118
|
|
@@ -42,6 +123,11 @@
|
|
42
123
|
**Implemented enhancements:**
|
43
124
|
|
44
125
|
- Add environment variable to mirror `cleanup\_preserved\_jobs --before-seconds-ago=SECONDS` [\#110](https://github.com/bensheldon/good_job/issues/110)
|
126
|
+
- Allow env variable config for cleanups [\#114](https://github.com/bensheldon/good_job/pull/114) ([gadimbaylisahil](https://github.com/gadimbaylisahil))
|
127
|
+
|
128
|
+
**Fixed bugs:**
|
129
|
+
|
130
|
+
- Better table name detection for Job queries [\#119](https://github.com/bensheldon/good_job/pull/119) ([gadimbaylisahil](https://github.com/gadimbaylisahil))
|
45
131
|
|
46
132
|
**Closed issues:**
|
47
133
|
|
@@ -52,9 +138,7 @@
|
|
52
138
|
**Merged pull requests:**
|
53
139
|
|
54
140
|
- Remove unused PgLocks class [\#120](https://github.com/bensheldon/good_job/pull/120) ([gadimbaylisahil](https://github.com/gadimbaylisahil))
|
55
|
-
- Better table name detection for Job queries [\#119](https://github.com/bensheldon/good_job/pull/119) ([gadimbaylisahil](https://github.com/gadimbaylisahil))
|
56
141
|
- Fix readme CommandLine option links [\#115](https://github.com/bensheldon/good_job/pull/115) ([gadimbaylisahil](https://github.com/gadimbaylisahil))
|
57
|
-
- Allow env variable config for cleanups [\#114](https://github.com/bensheldon/good_job/pull/114) ([gadimbaylisahil](https://github.com/gadimbaylisahil))
|
58
142
|
- Have YARD render markdown files with GFM \(Github Flavored Markdown\) [\#113](https://github.com/bensheldon/good_job/pull/113) ([bensheldon](https://github.com/bensheldon))
|
59
143
|
- Add markdownlint to lint readme [\#109](https://github.com/bensheldon/good_job/pull/109) ([bensheldon](https://github.com/bensheldon))
|
60
144
|
- Remove unused method in PgLocks [\#107](https://github.com/bensheldon/good_job/pull/107) ([gadimbaylisahil](https://github.com/gadimbaylisahil))
|
@@ -71,6 +155,7 @@
|
|
71
155
|
**Merged pull requests:**
|
72
156
|
|
73
157
|
- stop depending on all rails libs [\#104](https://github.com/bensheldon/good_job/pull/104) ([thilo](https://github.com/thilo))
|
158
|
+
- Use more ActiveRecord in Lockable and not connection.execute [\#102](https://github.com/bensheldon/good_job/pull/102) ([bensheldon](https://github.com/bensheldon))
|
74
159
|
|
75
160
|
## [v1.2.2](https://github.com/bensheldon/good_job/tree/v1.2.2) (2020-08-27)
|
76
161
|
|
@@ -91,7 +176,6 @@
|
|
91
176
|
|
92
177
|
**Merged pull requests:**
|
93
178
|
|
94
|
-
- Use more ActiveRecord in Lockable and not connection.execute [\#102](https://github.com/bensheldon/good_job/pull/102) ([bensheldon](https://github.com/bensheldon))
|
95
179
|
- Run CI tests on Ruby 2.5, 2.6, and 2.7 [\#101](https://github.com/bensheldon/good_job/pull/101) ([arku](https://github.com/arku))
|
96
180
|
- Return to using executor.wrap around Scheduler execution task [\#99](https://github.com/bensheldon/good_job/pull/99) ([bensheldon](https://github.com/bensheldon))
|
97
181
|
- Fix Ruby 2.7 keyword arguments warning [\#98](https://github.com/bensheldon/good_job/pull/98) ([arku](https://github.com/arku))
|
@@ -354,7 +438,6 @@
|
|
354
438
|
- Add pg gem as explicit dependency [\#13](https://github.com/bensheldon/good_job/pull/13) ([bensheldon](https://github.com/bensheldon))
|
355
439
|
- Bump nokogiri from 1.10.7 to 1.10.9 [\#12](https://github.com/bensheldon/good_job/pull/12) ([dependabot[bot]](https://github.com/apps/dependabot))
|
356
440
|
- Add Appraisal with tests for Rails 5.1, 5.2, 6.0 [\#11](https://github.com/bensheldon/good_job/pull/11) ([bensheldon](https://github.com/bensheldon))
|
357
|
-
- Use Rails.logger and ActiveSupport::Notifications for logging instead of puts [\#10](https://github.com/bensheldon/good_job/pull/10) ([bensheldon](https://github.com/bensheldon))
|
358
441
|
|
359
442
|
## [v0.2.0](https://github.com/bensheldon/good_job/tree/v0.2.0) (2020-03-06)
|
360
443
|
|
@@ -362,6 +445,7 @@
|
|
362
445
|
|
363
446
|
**Merged pull requests:**
|
364
447
|
|
448
|
+
- Use Rails.logger and ActiveSupport::Notifications for logging instead of puts [\#10](https://github.com/bensheldon/good_job/pull/10) ([bensheldon](https://github.com/bensheldon))
|
365
449
|
- Remove minitest files [\#9](https://github.com/bensheldon/good_job/pull/9) ([bensheldon](https://github.com/bensheldon))
|
366
450
|
- Use scheduled\_at and priority for scheduling [\#8](https://github.com/bensheldon/good_job/pull/8) ([bensheldon](https://github.com/bensheldon))
|
367
451
|
- Create Github Action workflow for PRs and Issues [\#7](https://github.com/bensheldon/good_job/pull/7) ([bensheldon](https://github.com/bensheldon))
|
data/README.md
CHANGED
@@ -1,5 +1,8 @@
|
|
1
1
|
# GoodJob
|
2
2
|
|
3
|
+
[](https://rubygems.org/gems/good_job)
|
4
|
+
[](https://github.com/bensheldon/good_job/actions)
|
5
|
+
|
3
6
|
GoodJob is a multithreaded, Postgres-based, ActiveJob backend for Ruby on Rails.
|
4
7
|
|
5
8
|
**Inspired by [Delayed::Job](https://github.com/collectiveidea/delayed_job) and [Que](https://github.com/que-rb/que), GoodJob is designed for maximum compatibility with Ruby on Rails, ActiveJob, and Postgres to be simple and performant for most workloads.**
|
@@ -33,7 +36,8 @@ For more of the story of GoodJob, read the [introductory blog post](https://isla
|
|
33
36
|
- [`good_job cleanup_preserved_jobs`](#good_job-cleanup_preserved_jobs)
|
34
37
|
- [Adapter options](#adapter-options)
|
35
38
|
- [Global options](#global-options)
|
36
|
-
- [
|
39
|
+
- [Dashboard](#dashboard)
|
40
|
+
- [Go deeper](#go-deeper)
|
37
41
|
- [Exceptions, retries, and reliability](#exceptions-retries-and-reliability)
|
38
42
|
- [Exceptions](#exceptions)
|
39
43
|
- [Retries](#retries)
|
@@ -41,12 +45,12 @@ For more of the story of GoodJob, read the [introductory blog post](https://isla
|
|
41
45
|
- [Timeouts](#timeouts)
|
42
46
|
- [Optimize queues, threads, and processes](#optimize-queues-threads-and-processes)
|
43
47
|
- [Database connections](#database-connections)
|
44
|
-
- [
|
45
|
-
- [
|
46
|
-
- [
|
47
|
-
- [
|
48
|
+
- [Execute jobs async / in-process](#execute-jobs-async--in-process)
|
49
|
+
- [Migrate to GoodJob from a different ActiveJob backend](#migrate-to-goodjob-from-a-different-activejob-backend)
|
50
|
+
- [Monitor and preserve worked jobs](#monitor-and-preserve-worked-jobs)
|
51
|
+
- [Contribute](#contribute)
|
48
52
|
- [Gem development](#gem-development)
|
49
|
-
- [
|
53
|
+
- [Release](#release)
|
50
54
|
- [License](#license)
|
51
55
|
|
52
56
|
## Set up
|
@@ -209,7 +213,7 @@ Good Job’s general behavior can also be configured via several attributes dire
|
|
209
213
|
|
210
214
|
- **`GoodJob.logger`** ([Rails Logger](https://api.rubyonrails.org/classes/ActiveSupport/Logger.html)) lets you set a custom logger for GoodJob. It should be an instance of a Rails `Logger`.
|
211
215
|
- **`GoodJob.preserve_job_records`** (boolean) keeps job records in your database even after jobs are completed. (Default: `false`)
|
212
|
-
- **`GoodJob.
|
216
|
+
- **`GoodJob.retry_on_unhandled_error`** (boolean) causes jobs to be re-queued and retried if they raise an instance of `StandardError`. Instances of `Exception`, like SIGINT, will *always* be retried, regardless of this attribute’s value. (Default: `true`)
|
213
217
|
- **`GoodJob.on_thread_error`** (proc, lambda, or callable) will be called when an Exception. It can be useful for logging errors to bug tracking services, like Sentry or Airbrake.
|
214
218
|
|
215
219
|
You’ll generally want to configure these in `config/initializers/good_job.rb`, like so:
|
@@ -217,11 +221,56 @@ You’ll generally want to configure these in `config/initializers/good_job.rb`,
|
|
217
221
|
```ruby
|
218
222
|
# config/initializers/good_job.rb
|
219
223
|
GoodJob.preserve_job_records = true
|
220
|
-
GoodJob.
|
224
|
+
GoodJob.retry_on_unhandled_error = false
|
221
225
|
GoodJob.on_thread_error = -> (exception) { Raven.capture_exception(exception) }
|
222
226
|
```
|
223
227
|
|
224
|
-
|
228
|
+
### Dashboard
|
229
|
+
|
230
|
+
_🚧 GoodJob's dashboard is a work in progress. Please contribute ideas and code on [Github](https://github.com/bensheldon/good_job/issues)._
|
231
|
+
|
232
|
+
GoodJob includes a Dashboard as a mountable `Rails::Engine`.
|
233
|
+
|
234
|
+
1. Explicitly require the Engine code at the top of your `config/application.rb` file, immediately after Rails is required. This is necessary because the mountable engine is an optional feature of GoodJob.
|
235
|
+
|
236
|
+
```ruby
|
237
|
+
# config/application.rb
|
238
|
+
require_relative 'boot'
|
239
|
+
|
240
|
+
require 'rails/all'
|
241
|
+
require 'good_job/engine' # <= Add this line
|
242
|
+
# ...
|
243
|
+
```
|
244
|
+
|
245
|
+
1. Mount the engine in your `config/routes.rb` file. The following will mount it at `http://example.com/good_job`.
|
246
|
+
|
247
|
+
```ruby
|
248
|
+
# config/routes.rb
|
249
|
+
# ...
|
250
|
+
mount GoodJob::Engine => 'good_job'
|
251
|
+
```
|
252
|
+
|
253
|
+
Because jobs can potentially contain sensitive information, you should authorize access. For example, using Devise's `authenticate` helper, that might look like:
|
254
|
+
|
255
|
+
```ruby
|
256
|
+
# config/routes.rb
|
257
|
+
# ...
|
258
|
+
authenticate :user, ->(user) { user.admin? } do
|
259
|
+
mount GoodJob::Engine => 'good_job'
|
260
|
+
end
|
261
|
+
```
|
262
|
+
|
263
|
+
Another option is using basic auth like this:
|
264
|
+
|
265
|
+
```ruby
|
266
|
+
# config/initializers/good_job.rb
|
267
|
+
GoodJob::Engine.middleware.use(Rack::Auth::Basic) do |username, password|
|
268
|
+
ActiveSupport::SecurityUtils.secure_compare(Rails.application.credentials.good_job_username, username) &&
|
269
|
+
ActiveSupport::SecurityUtils.secure_compare(Rails.application.credentials.good_job_password, password)
|
270
|
+
end
|
271
|
+
```
|
272
|
+
|
273
|
+
## Go deeper
|
225
274
|
|
226
275
|
### Exceptions, retries, and reliability
|
227
276
|
|
@@ -255,7 +304,7 @@ When using `retry_on` with _a limited number of retries_, the final exception wi
|
|
255
304
|
|
256
305
|
```ruby
|
257
306
|
# config/initializers/good_job.rb
|
258
|
-
GoodJob.
|
307
|
+
GoodJob.retry_on_unhandled_error = false
|
259
308
|
```
|
260
309
|
|
261
310
|
Alternatively, pass a block to `retry_on` to handle the final exception instead of raising it to GoodJob:
|
@@ -380,7 +429,7 @@ Each GoodJob execution thread requires its own database connection that is autom
|
|
380
429
|
pool: <%= [ENV.fetch("RAILS_MAX_THREADS", 5).to_i, ENV.fetch("GOOD_JOB_MAX_THREADS", 4).to_i].max %>
|
381
430
|
```
|
382
431
|
|
383
|
-
###
|
432
|
+
### Execute jobs async / in-process
|
384
433
|
|
385
434
|
GoodJob can execute jobs "async" in the same process as the webserver (e.g. `bin/rail s`). GoodJob's async execution mode offers benefits of economy by not requiring a separate job worker process, but with the tradeoff of increased complexity. Async mode can be configured in two ways:
|
386
435
|
|
@@ -431,7 +480,7 @@ Depending on your application configuration, you may need to take additional ste
|
|
431
480
|
|
432
481
|
GoodJob is compatible with Puma's `preload_app!` method.
|
433
482
|
|
434
|
-
###
|
483
|
+
### Migrate to GoodJob from a different ActiveJob backend
|
435
484
|
|
436
485
|
If your application is already using an ActiveJob backend, you will need to install GoodJob to enqueue and perform newly created jobs _and_ finish performing pre-existing jobs on the previous backend.
|
437
486
|
|
@@ -455,7 +504,7 @@ If your application is already using an ActiveJob backend, you will need to inst
|
|
455
504
|
|
456
505
|
1. Once you are confident that no unperformed jobs remain in the previous ActiveJob backend, code and configuration for that backend can be completely removed.
|
457
506
|
|
458
|
-
###
|
507
|
+
### Monitor and preserve worked jobs
|
459
508
|
|
460
509
|
GoodJob is fully instrumented with [`ActiveSupport::Notifications`](https://edgeguides.rubyonrails.org/active_support_instrumentation.html#introduction-to-instrumentation).
|
461
510
|
|
@@ -482,7 +531,7 @@ It is also necessary to delete these preserved jobs from the database after a ce
|
|
482
531
|
$ bundle exec good_job cleanup_preserved_jobs --before-seconds-ago=86400
|
483
532
|
```
|
484
533
|
|
485
|
-
##
|
534
|
+
## Contribute
|
486
535
|
|
487
536
|
Contributions are welcomed and appreciated 🙏
|
488
537
|
|
@@ -527,7 +576,7 @@ $ bundle install
|
|
527
576
|
# => Using good_job 0.1.0 from https://github.com/bensheldon/good_job.git (at /Users/You/Projects/good_job@dc57fb0)
|
528
577
|
```
|
529
578
|
|
530
|
-
###
|
579
|
+
### Release
|
531
580
|
|
532
581
|
Package maintainers can release this gem by running:
|
533
582
|
|
@@ -1,10 +1,59 @@
|
|
1
1
|
module GoodJob
|
2
2
|
class DashboardsController < GoodJob::BaseController
|
3
|
-
|
4
|
-
|
3
|
+
class JobFilter
|
4
|
+
attr_accessor :params
|
5
|
+
|
6
|
+
def initialize(params)
|
7
|
+
@params = params
|
8
|
+
end
|
9
|
+
|
10
|
+
def last
|
11
|
+
@_last ||= jobs.last
|
12
|
+
end
|
13
|
+
|
14
|
+
def jobs
|
15
|
+
sql = GoodJob::Job.display_all(after_scheduled_at: params[:after_scheduled_at], after_id: params[:after_id])
|
5
16
|
.limit(params.fetch(:limit, 10))
|
17
|
+
if params[:job_class] # rubocop:disable Style/IfUnlessModifier
|
18
|
+
sql = sql.where("serialized_params->>'job_class' = ?", params[:job_class])
|
19
|
+
end
|
20
|
+
if params[:state]
|
21
|
+
case params[:state]
|
22
|
+
when 'finished'
|
23
|
+
sql = sql.finished
|
24
|
+
when 'unfinished'
|
25
|
+
sql = sql.unfinished
|
26
|
+
when 'errors'
|
27
|
+
sql = sql.where.not(error: nil)
|
28
|
+
end
|
29
|
+
end
|
30
|
+
sql
|
31
|
+
end
|
32
|
+
|
33
|
+
def states
|
34
|
+
{
|
35
|
+
'finished' => GoodJob::Job.finished.count,
|
36
|
+
'unfinished' => GoodJob::Job.unfinished.count,
|
37
|
+
'errors' => GoodJob::Job.where.not(error: nil).count,
|
38
|
+
}
|
39
|
+
end
|
40
|
+
|
41
|
+
def job_classes
|
42
|
+
GoodJob::Job.group("serialized_params->>'job_class'").count
|
43
|
+
end
|
44
|
+
|
45
|
+
def to_query(override)
|
46
|
+
{
|
47
|
+
state: params[:state],
|
48
|
+
job_class: params[:job_class],
|
49
|
+
}.merge(override).delete_if { |_, v| v.nil? }.to_query
|
50
|
+
end
|
51
|
+
end
|
52
|
+
|
53
|
+
def index
|
54
|
+
@filter = JobFilter.new(params)
|
6
55
|
|
7
|
-
job_data = GoodJob::Job.connection.exec_query Arel.sql(<<~SQL)
|
56
|
+
job_data = GoodJob::Job.connection.exec_query Arel.sql(<<~SQL.squish)
|
8
57
|
SELECT *
|
9
58
|
FROM generate_series(
|
10
59
|
date_trunc('hour', NOW() - '1 day'::interval),
|
@@ -2,13 +2,36 @@
|
|
2
2
|
<%= render 'shared/chart', chart_data: @chart %>
|
3
3
|
</div>
|
4
4
|
|
5
|
-
|
6
|
-
|
5
|
+
<div class='card mb-2'>
|
6
|
+
<div class='card-body d-flex flex-wrap'>
|
7
|
+
<div class='mr-4'>
|
8
|
+
<small>Filter by job class</small>
|
9
|
+
<br>
|
10
|
+
<% @filter.job_classes.each do |name, count| %>
|
11
|
+
<a href='<%= request.path + "?#{@filter.to_query(job_class: name)}" %>' class='btn btn-sm btn-outline-secondary <%= "active" if params[:job_class] == name %>'>
|
12
|
+
<%= name %> (<%= count %>)
|
13
|
+
</a>
|
14
|
+
<% end %>
|
15
|
+
</div>
|
16
|
+
<div>
|
17
|
+
<small>Filter by state</small>
|
18
|
+
<br>
|
19
|
+
<% @filter.states.each do |name, count| %>
|
20
|
+
<a href='<%= request.path + "?#{@filter.to_query(state: name)}" %>' class='btn btn-sm btn-outline-secondary <%= "active" if params[:state] == name %>'>
|
21
|
+
<%= name %> (<%= count %>)
|
22
|
+
</a>
|
23
|
+
<% end %>
|
24
|
+
</div>
|
25
|
+
</div>
|
26
|
+
</div>
|
27
|
+
|
28
|
+
<% if @filter.jobs.present? %>
|
29
|
+
<%= render 'shared/jobs_table', jobs: @filter.jobs %>
|
7
30
|
|
8
31
|
<nav aria-label="Job pagination">
|
9
32
|
<ul class="pagination">
|
10
33
|
<li class="page-item">
|
11
|
-
<%= link_to({ after_scheduled_at: (@
|
34
|
+
<%= link_to({ after_scheduled_at: (@filter.last.scheduled_at || @filter.last.created_at), after_id: @filter.last.id }, class: "page-link") do %>
|
12
35
|
Next jobs <span aria-hidden="true">»</span>
|
13
36
|
<% end %>
|
14
37
|
</li>
|
@@ -18,7 +18,7 @@
|
|
18
18
|
</head>
|
19
19
|
<body>
|
20
20
|
<nav class="navbar navbar-expand-lg navbar-light bg-light">
|
21
|
-
<div class="container">
|
21
|
+
<div class="container-fluid">
|
22
22
|
<%= link_to "GoodJob 👍", root_path, class: 'navbar-brand mb-0 h1' %>
|
23
23
|
<button class="navbar-toggler" type="button" data-toggle="collapse" data-target="#navbarSupportedContent" aria-controls="navbarSupportedContent" aria-expanded="false" aria-label="Toggle navigation">
|
24
24
|
<span class="navbar-toggler-icon"></span>
|
@@ -27,23 +27,34 @@
|
|
27
27
|
<div class="collapse navbar-collapse" id="navbarSupportedContent">
|
28
28
|
<ul class="navbar-nav mr-auto">
|
29
29
|
<li class="nav-item">
|
30
|
-
<%= link_to
|
31
|
-
|
32
|
-
|
33
|
-
<%= link_to "Upcoming Jobs", 'todo', class: ["nav-link", ("active" if current_page?('todo'))] %>
|
34
|
-
</li>
|
35
|
-
<li class="nav-item">
|
36
|
-
<%= link_to "Finished Jobs", 'todo', class: ["nav-link", ("active" if current_page?('todo'))] %>
|
37
|
-
</li>
|
38
|
-
<li class="nav-item">
|
39
|
-
<%= link_to "Errored Jobs", 'todo', class: ["nav-link", ("active" if current_page?('todo'))] %>
|
30
|
+
<%= link_to root_path, class: ["nav-link", ("active" if current_page?(root_path))] do %>
|
31
|
+
All jobs <span class="badge badge-secondary">More views coming soon</span>
|
32
|
+
<% end %>
|
40
33
|
</li>
|
34
|
+
|
35
|
+
<!-- Coming Soon
|
36
|
+
<li class="nav-item">
|
37
|
+
<%= link_to "Upcoming Jobs", 'todo', class: ["nav-link", ("active" if current_page?('todo'))] %>
|
38
|
+
</li>
|
39
|
+
<li class="nav-item">
|
40
|
+
<%= link_to "Finished Jobs", 'todo', class: ["nav-link", ("active" if current_page?('todo'))] %>
|
41
|
+
</li>
|
42
|
+
<li class="nav-item">
|
43
|
+
<%= link_to "Errored Jobs", 'todo', class: ["nav-link", ("active" if current_page?('todo'))] %>
|
44
|
+
</li>
|
45
|
+
-->
|
41
46
|
</ul>
|
42
47
|
</div>
|
43
48
|
</div>
|
44
49
|
</nav>
|
45
50
|
|
46
|
-
<div class="container">
|
51
|
+
<div class="container-fluid">
|
52
|
+
<div class="card border-warning text-dark my-3">
|
53
|
+
<div class="card-body">
|
54
|
+
<p class="card-text">🚧 GoodJob's dashboard is a work in progress. Please contribute ideas and code on <a href="https://github.com/bensheldon/good_job/issues" target="_blank" rel="nofollow noopener noreferrer">Github</a>.</p>
|
55
|
+
</div>
|
56
|
+
</div>
|
57
|
+
|
47
58
|
<%= yield %>
|
48
59
|
</div>
|
49
60
|
</body>
|
@@ -1,13 +1,13 @@
|
|
1
1
|
<div class="table-responsive">
|
2
|
-
<table class="table table-bordered table-hover">
|
2
|
+
<table class="table table-bordered table-hover table-sm">
|
3
3
|
<thead>
|
4
4
|
<th>GoodJob ID</th>
|
5
5
|
<th>ActiveJob ID</th>
|
6
6
|
<th>Job Class</th>
|
7
7
|
<th>Queue</th>
|
8
8
|
<th>Scheduled At</th>
|
9
|
-
<th>ActiveJob Params</th>
|
10
9
|
<th>Error</th>
|
10
|
+
<th>ActiveJob Params</th>
|
11
11
|
</thead>
|
12
12
|
<tbody>
|
13
13
|
<% jobs.each do |job| %>
|
@@ -17,8 +17,8 @@
|
|
17
17
|
<td><%= job.serialized_params['job_class'] %></td>
|
18
18
|
<td><%= job.queue_name %></td>
|
19
19
|
<td><%= job.scheduled_at || job.created_at %></td>
|
20
|
-
<td><%= job.serialized_params %></td>
|
21
20
|
<td><%= job.error %></td>
|
21
|
+
<td><pre><%= JSON.pretty_generate(job.serialized_params) %></pre></td>
|
22
22
|
</tr>
|
23
23
|
<% end %>
|
24
24
|
</tbody>
|
data/lib/good_job.rb
CHANGED
@@ -32,18 +32,35 @@ module GoodJob
|
|
32
32
|
# Whether to preserve job records in the database after they have finished (default: +false+).
|
33
33
|
# By default, GoodJob deletes job records after the job is completed successfully.
|
34
34
|
# If you want to preserve jobs for latter inspection, set this to +true+.
|
35
|
+
# If you want to preserve only jobs that finished with error for latter inspection, set this to +:on_unhandled_error+.
|
35
36
|
# If +true+, you will need to clean out jobs using the +good_job cleanup_preserved_jobs+ CLI command.
|
36
37
|
# @return [Boolean]
|
37
38
|
mattr_accessor :preserve_job_records, default: false
|
38
39
|
|
39
|
-
# @!attribute [rw]
|
40
|
+
# @!attribute [rw] retry_on_unhandled_error
|
40
41
|
# @!scope class
|
41
42
|
# Whether to re-perform a job when a type of +StandardError+ is raised to GoodJob (default: +true+).
|
42
43
|
# If +true+, causes jobs to be re-queued and retried if they raise an instance of +StandardError+.
|
43
44
|
# If +false+, jobs will be discarded or marked as finished if they raise an instance of +StandardError+.
|
44
45
|
# Instances of +Exception+, like +SIGINT+, will *always* be retried, regardless of this attribute's value.
|
45
46
|
# @return [Boolean]
|
46
|
-
mattr_accessor :
|
47
|
+
mattr_accessor :retry_on_unhandled_error, default: true
|
48
|
+
|
49
|
+
# @deprecated Use {GoodJob#retry_on_unhandled_error} instead.
|
50
|
+
def self.reperform_jobs_on_standard_error
|
51
|
+
ActiveSupport::Deprecation.warn(
|
52
|
+
"Calling 'GoodJob.reperform_jobs_on_standard_error' is deprecated. Please use 'retry_on_unhandled_error'"
|
53
|
+
)
|
54
|
+
retry_on_unhandled_error
|
55
|
+
end
|
56
|
+
|
57
|
+
# @deprecated Use {GoodJob#retry_on_unhandled_error=} instead.
|
58
|
+
def self.reperform_jobs_on_standard_error=(value)
|
59
|
+
ActiveSupport::Deprecation.warn(
|
60
|
+
"Setting 'GoodJob.reperform_jobs_on_standard_error=' is deprecated. Please use 'retry_on_unhandled_error='"
|
61
|
+
)
|
62
|
+
self.retry_on_unhandled_error = value
|
63
|
+
end
|
47
64
|
|
48
65
|
# @!attribute [rw] on_thread_error
|
49
66
|
# @!scope class
|
data/lib/good_job/adapter.rb
CHANGED
@@ -43,8 +43,10 @@ module GoodJob
|
|
43
43
|
|
44
44
|
if @execution_mode == :async # rubocop:disable Style/GuardClause
|
45
45
|
@notifier = notifier || GoodJob::Notifier.new
|
46
|
+
@poller = GoodJob::Poller.new(poll_interval: configuration.poll_interval)
|
46
47
|
@scheduler = scheduler || GoodJob::Scheduler.from_configuration(configuration)
|
47
48
|
@notifier.recipients << [@scheduler, :create_thread]
|
49
|
+
@poller.recipients << [@scheduler, :create_thread]
|
48
50
|
end
|
49
51
|
end
|
50
52
|
|
@@ -88,6 +90,7 @@ module GoodJob
|
|
88
90
|
# @return [void]
|
89
91
|
def shutdown(wait: true)
|
90
92
|
@notifier&.shutdown(wait: wait)
|
93
|
+
@poller&.shutdown(wait: wait)
|
91
94
|
@scheduler&.shutdown(wait: wait)
|
92
95
|
end
|
93
96
|
|
data/lib/good_job/cli.rb
CHANGED
@@ -42,14 +42,16 @@ module GoodJob
|
|
42
42
|
method_option :poll_interval,
|
43
43
|
type: :numeric,
|
44
44
|
banner: 'SECONDS',
|
45
|
-
desc: "Interval between polls for available jobs in seconds (env var: GOOD_JOB_POLL_INTERVAL, default:
|
45
|
+
desc: "Interval between polls for available jobs in seconds (env var: GOOD_JOB_POLL_INTERVAL, default: 5)"
|
46
46
|
def start
|
47
47
|
set_up_application!
|
48
|
+
configuration = GoodJob::Configuration.new(options)
|
48
49
|
|
49
50
|
notifier = GoodJob::Notifier.new
|
50
|
-
|
51
|
+
poller = GoodJob::Poller.new(poll_interval: configuration.poll_interval)
|
51
52
|
scheduler = GoodJob::Scheduler.from_configuration(configuration)
|
52
53
|
notifier.recipients << [scheduler, :create_thread]
|
54
|
+
poller.recipients << [scheduler, :create_thread]
|
53
55
|
|
54
56
|
@stop_good_job_executable = false
|
55
57
|
%w[INT TERM].each do |signal|
|
@@ -62,6 +64,7 @@ module GoodJob
|
|
62
64
|
end
|
63
65
|
|
64
66
|
notifier.shutdown
|
67
|
+
poller.shutdown
|
65
68
|
scheduler.shutdown
|
66
69
|
end
|
67
70
|
|
@@ -5,6 +5,13 @@ module GoodJob
|
|
5
5
|
# set options to get the final values for each option.
|
6
6
|
#
|
7
7
|
class Configuration
|
8
|
+
# Default number of threads to use per {Scheduler}
|
9
|
+
DEFAULT_MAX_THREADS = 5
|
10
|
+
# Default number of seconds between polls for jobs
|
11
|
+
DEFAULT_POLL_INTERVAL = 5
|
12
|
+
# Default number of seconds to preserve jobs for {CLI#cleanup_preserved_jobs}
|
13
|
+
DEFAULT_CLEANUP_PRESERVED_JOBS_BEFORE_SECONDS_AGO = 24 * 60 * 60
|
14
|
+
|
8
15
|
# @!attribute [r] options
|
9
16
|
# The options that were explicitly set when initializing +Configuration+.
|
10
17
|
# @return [Hash]
|
@@ -69,7 +76,7 @@ module GoodJob
|
|
69
76
|
options[:max_threads] ||
|
70
77
|
env['GOOD_JOB_MAX_THREADS'] ||
|
71
78
|
env['RAILS_MAX_THREADS'] ||
|
72
|
-
|
79
|
+
DEFAULT_MAX_THREADS
|
73
80
|
).to_i
|
74
81
|
end
|
75
82
|
|
@@ -92,7 +99,7 @@ module GoodJob
|
|
92
99
|
(
|
93
100
|
options[:poll_interval] ||
|
94
101
|
env['GOOD_JOB_POLL_INTERVAL'] ||
|
95
|
-
|
102
|
+
DEFAULT_POLL_INTERVAL
|
96
103
|
).to_i
|
97
104
|
end
|
98
105
|
|
@@ -100,7 +107,7 @@ module GoodJob
|
|
100
107
|
(
|
101
108
|
options[:before_seconds_ago] ||
|
102
109
|
env['GOOD_JOB_CLEANUP_PRESERVED_JOBS_BEFORE_SECONDS_AGO'] ||
|
103
|
-
|
110
|
+
DEFAULT_CLEANUP_PRESERVED_JOBS_BEFORE_SECONDS_AGO
|
104
111
|
).to_i
|
105
112
|
end
|
106
113
|
end
|
data/lib/good_job/job.rb
CHANGED
@@ -189,37 +189,31 @@ module GoodJob
|
|
189
189
|
self.performed_at = Time.current
|
190
190
|
save! if GoodJob.preserve_job_records
|
191
191
|
|
192
|
-
result,
|
192
|
+
result, unhandled_error = execute
|
193
193
|
|
194
|
-
|
195
|
-
|
196
|
-
|
197
|
-
error = nil
|
198
|
-
if rescued_error
|
199
|
-
error = rescued_error
|
200
|
-
elsif result.is_a?(Exception)
|
201
|
-
error = result
|
194
|
+
result_error = nil
|
195
|
+
if result.is_a?(Exception)
|
196
|
+
result_error = result
|
202
197
|
result = nil
|
203
|
-
elsif retry_or_discard_error
|
204
|
-
error = retry_or_discard_error
|
205
198
|
end
|
206
199
|
|
207
|
-
|
208
|
-
|
200
|
+
job_error = unhandled_error ||
|
201
|
+
result_error ||
|
202
|
+
GoodJob::CurrentExecution.error_on_retry ||
|
203
|
+
GoodJob::CurrentExecution.error_on_discard
|
209
204
|
|
210
|
-
|
205
|
+
self.error = "#{job_error.class}: #{job_error.message}" if job_error
|
206
|
+
|
207
|
+
if unhandled_error && GoodJob.retry_on_unhandled_error
|
211
208
|
save!
|
212
|
-
|
209
|
+
elsif GoodJob.preserve_job_records == true || (unhandled_error && GoodJob.preserve_job_records == :on_unhandled_error)
|
213
210
|
self.finished_at = Time.current
|
214
|
-
|
215
|
-
|
216
|
-
|
217
|
-
else
|
218
|
-
destroy!
|
219
|
-
end
|
211
|
+
save!
|
212
|
+
else
|
213
|
+
destroy!
|
220
214
|
end
|
221
215
|
|
222
|
-
[result,
|
216
|
+
[result, job_error]
|
223
217
|
end
|
224
218
|
|
225
219
|
private
|
data/lib/good_job/lockable.rb
CHANGED
@@ -56,7 +56,7 @@ module GoodJob
|
|
56
56
|
# @example Get the records that have a session awaiting a lock:
|
57
57
|
# MyLockableRecord.joins_advisory_locks.where("pg_locks.granted = ?", false)
|
58
58
|
scope :joins_advisory_locks, (lambda do
|
59
|
-
join_sql = <<~SQL
|
59
|
+
join_sql = <<~SQL.squish
|
60
60
|
LEFT JOIN pg_locks ON pg_locks.locktype = 'advisory'
|
61
61
|
AND pg_locks.objsubid = 1
|
62
62
|
AND pg_locks.classid = ('x' || substr(md5(:table_name || #{quoted_table_name}.#{quoted_primary_key}::text), 1, 16))::bit(32)::int
|
@@ -140,10 +140,10 @@ module GoodJob
|
|
140
140
|
# all remaining locks).
|
141
141
|
# @return [Boolean] whether the lock was acquired.
|
142
142
|
def advisory_lock
|
143
|
-
where_sql = <<~SQL
|
143
|
+
where_sql = <<~SQL.squish
|
144
144
|
pg_try_advisory_lock(('x' || substr(md5(:table_name || :id::text), 1, 16))::bit(64)::bigint)
|
145
145
|
SQL
|
146
|
-
self.class.unscoped.
|
146
|
+
self.class.unscoped.exists?([where_sql, { table_name: self.class.table_name, id: send(self.class.primary_key) }])
|
147
147
|
end
|
148
148
|
|
149
149
|
# Releases an advisory lock on this record if it is locked by this database
|
@@ -151,10 +151,10 @@ module GoodJob
|
|
151
151
|
# {#advisory_unlock} and {#advisory_lock} the same number of times.
|
152
152
|
# @return [Boolean] whether the lock was released.
|
153
153
|
def advisory_unlock
|
154
|
-
where_sql = <<~SQL
|
154
|
+
where_sql = <<~SQL.squish
|
155
155
|
pg_advisory_unlock(('x' || substr(md5(:table_name || :id::text), 1, 16))::bit(64)::bigint)
|
156
156
|
SQL
|
157
|
-
self.class.unscoped.
|
157
|
+
self.class.unscoped.exists?([where_sql, { table_name: self.class.table_name, id: send(self.class.primary_key) }])
|
158
158
|
end
|
159
159
|
|
160
160
|
# Acquires an advisory lock on this record or raises
|
@@ -191,13 +191,13 @@ module GoodJob
|
|
191
191
|
# Tests whether this record has an advisory lock on it.
|
192
192
|
# @return [Boolean]
|
193
193
|
def advisory_locked?
|
194
|
-
self.class.unscoped.advisory_locked.
|
194
|
+
self.class.unscoped.advisory_locked.exists?(id: send(self.class.primary_key))
|
195
195
|
end
|
196
196
|
|
197
197
|
# Tests whether this record is locked by the current database session.
|
198
198
|
# @return [Boolean]
|
199
199
|
def owns_advisory_lock?
|
200
|
-
self.class.unscoped.owns_advisory_locked.
|
200
|
+
self.class.unscoped.owns_advisory_locked.exists?(id: send(self.class.primary_key))
|
201
201
|
end
|
202
202
|
|
203
203
|
# Releases all advisory locks on the record that are held by the current
|
@@ -25,8 +25,7 @@ module GoodJob
|
|
25
25
|
end
|
26
26
|
|
27
27
|
# @macro notification_responder
|
28
|
-
def
|
29
|
-
# FIXME: This method does not match any good_job notifications.
|
28
|
+
def finished_timer_task(event)
|
30
29
|
exception = event.payload[:error]
|
31
30
|
return unless exception
|
32
31
|
|
@@ -36,8 +35,7 @@ module GoodJob
|
|
36
35
|
end
|
37
36
|
|
38
37
|
# @macro notification_responder
|
39
|
-
def
|
40
|
-
# FIXME: This method does not match any good_job notifications.
|
38
|
+
def finished_job_task(event)
|
41
39
|
exception = event.payload[:error]
|
42
40
|
return unless exception
|
43
41
|
|
@@ -47,14 +45,13 @@ module GoodJob
|
|
47
45
|
end
|
48
46
|
|
49
47
|
# @macro notification_responder
|
50
|
-
def
|
48
|
+
def scheduler_create_pool(event)
|
51
49
|
max_threads = event.payload[:max_threads]
|
52
|
-
poll_interval = event.payload[:poll_interval]
|
53
50
|
performer_name = event.payload[:performer_name]
|
54
51
|
process_id = event.payload[:process_id]
|
55
52
|
|
56
53
|
info(tags: [process_id]) do
|
57
|
-
"GoodJob started scheduler with queues=#{performer_name} max_threads=#{max_threads}
|
54
|
+
"GoodJob started scheduler with queues=#{performer_name} max_threads=#{max_threads}."
|
58
55
|
end
|
59
56
|
end
|
60
57
|
|
@@ -168,12 +165,12 @@ module GoodJob
|
|
168
165
|
# @return [Logger]
|
169
166
|
def logger
|
170
167
|
@_logger ||= begin
|
171
|
-
|
172
|
-
|
173
|
-
|
174
|
-
|
175
|
-
|
176
|
-
|
168
|
+
logger = Logger.new(StringIO.new)
|
169
|
+
loggers.each do |each_logger|
|
170
|
+
logger.extend(ActiveSupport::Logger.broadcast(each_logger))
|
171
|
+
end
|
172
|
+
logger
|
173
|
+
end
|
177
174
|
end
|
178
175
|
|
179
176
|
# Reset {LogSubscriber.logger} and force it to rebuild a new shortcut to
|
@@ -194,11 +191,12 @@ module GoodJob
|
|
194
191
|
# @return [void]
|
195
192
|
def tag_logger(*tags, &block)
|
196
193
|
tags = tags.dup.unshift("GoodJob").compact
|
194
|
+
good_job_tag = ["ActiveJob"].freeze
|
197
195
|
|
198
196
|
self.class.loggers.inject(block) do |inner, each_logger|
|
199
197
|
if each_logger.respond_to?(:tagged)
|
200
198
|
tags_for_logger = if each_logger.formatter.current_tags.include?("ActiveJob")
|
201
|
-
|
199
|
+
good_job_tag + tags
|
202
200
|
else
|
203
201
|
tags
|
204
202
|
end
|
@@ -26,14 +26,23 @@ module GoodJob
|
|
26
26
|
# Delegates to {Scheduler#create_thread}.
|
27
27
|
def create_thread(state = nil)
|
28
28
|
results = []
|
29
|
-
|
30
|
-
|
29
|
+
|
30
|
+
if state
|
31
|
+
schedulers.any? do |scheduler|
|
32
|
+
scheduler.create_thread(state).tap { |result| results << result }
|
33
|
+
end
|
34
|
+
else
|
35
|
+
schedulers.each do |scheduler|
|
36
|
+
results << scheduler.create_thread(state)
|
37
|
+
end
|
31
38
|
end
|
32
39
|
|
33
|
-
if
|
40
|
+
if results.any?
|
34
41
|
true
|
35
|
-
|
36
|
-
|
42
|
+
elsif results.any? { |result| result == false }
|
43
|
+
false
|
44
|
+
else # rubocop:disable Style/EmptyElse
|
45
|
+
nil
|
37
46
|
end
|
38
47
|
end
|
39
48
|
end
|
data/lib/good_job/notifier.rb
CHANGED
@@ -34,7 +34,7 @@ module GoodJob # :nodoc:
|
|
34
34
|
# @param message [#to_json]
|
35
35
|
def self.notify(message)
|
36
36
|
connection = ActiveRecord::Base.connection
|
37
|
-
connection.exec_query <<~SQL
|
37
|
+
connection.exec_query <<~SQL.squish
|
38
38
|
NOTIFY #{CHANNEL}, #{connection.quote(message.to_json)}
|
39
39
|
SQL
|
40
40
|
end
|
@@ -75,7 +75,7 @@ module GoodJob # :nodoc:
|
|
75
75
|
# If +wait+ is +true+, the notifier will wait for background thread to shutdown.
|
76
76
|
# If +wait+ is +false+, this method will return immediately even though threads may still be running.
|
77
77
|
# Use {#shutdown?} to determine whether threads have stopped.
|
78
|
-
# @param wait [Boolean] Wait for actively executing
|
78
|
+
# @param wait [Boolean] Wait for actively executing threads to finish
|
79
79
|
# @return [void]
|
80
80
|
def shutdown(wait: true)
|
81
81
|
return unless @pool.running?
|
@@ -147,7 +147,7 @@ module GoodJob # :nodoc:
|
|
147
147
|
pg_conn.exec("SET application_name = #{pg_conn.escape_identifier(self.class.name)}")
|
148
148
|
yield pg_conn
|
149
149
|
ensure
|
150
|
-
ar_conn
|
150
|
+
ar_conn&.disconnect!
|
151
151
|
end
|
152
152
|
end
|
153
153
|
end
|
@@ -0,0 +1,94 @@
|
|
1
|
+
require 'concurrent/atomic/atomic_boolean'
|
2
|
+
|
3
|
+
module GoodJob # :nodoc:
|
4
|
+
#
|
5
|
+
# Pollers regularly wake up execution threads to check for new work.
|
6
|
+
#
|
7
|
+
class Poller
|
8
|
+
# Defaults for instance of Concurrent::TimerTask.
|
9
|
+
# The timer controls how and when sleeping threads check for new work.
|
10
|
+
DEFAULT_TIMER_OPTIONS = {
|
11
|
+
execution_interval: Configuration::DEFAULT_POLL_INTERVAL,
|
12
|
+
timeout_interval: 1,
|
13
|
+
run_now: true,
|
14
|
+
}.freeze
|
15
|
+
|
16
|
+
# @!attribute [r] instances
|
17
|
+
# @!scope class
|
18
|
+
# List of all instantiated Pollers in the current process.
|
19
|
+
# @return [array<GoodJob:Poller>]
|
20
|
+
cattr_reader :instances, default: [], instance_reader: false
|
21
|
+
|
22
|
+
def self.from_configuration(configuration)
|
23
|
+
GoodJob::Poller.new(poll_interval: configuration.poll_interval)
|
24
|
+
end
|
25
|
+
|
26
|
+
# List of recipients that will receive notifications.
|
27
|
+
# @return [Array<#call, Array(Object, Symbol)>]
|
28
|
+
attr_reader :recipients
|
29
|
+
|
30
|
+
# @param recipients [Array<#call, Array(Object, Symbol)>]
|
31
|
+
# @param poll_interval [Hash] number of seconds between polls
|
32
|
+
def initialize(*recipients, poll_interval: nil)
|
33
|
+
@recipients = Concurrent::Array.new(recipients)
|
34
|
+
|
35
|
+
@timer_options = DEFAULT_TIMER_OPTIONS.dup
|
36
|
+
@timer_options[:execution_interval] = poll_interval if poll_interval.present?
|
37
|
+
|
38
|
+
self.class.instances << self
|
39
|
+
|
40
|
+
create_pool
|
41
|
+
end
|
42
|
+
|
43
|
+
# Shut down the poller.
|
44
|
+
# If +wait+ is +true+, the poller will wait for background thread to shutdown.
|
45
|
+
# If +wait+ is +false+, this method will return immediately even though threads may still be running.
|
46
|
+
# Use {#shutdown?} to determine whether threads have stopped.
|
47
|
+
# @param wait [Boolean] Wait for actively executing threads to finish
|
48
|
+
# @return [void]
|
49
|
+
def shutdown(wait: true)
|
50
|
+
return unless @timer&.running?
|
51
|
+
|
52
|
+
@timer.shutdown
|
53
|
+
@timer.wait_for_termination if wait
|
54
|
+
end
|
55
|
+
|
56
|
+
# Tests whether the poller is shutdown.
|
57
|
+
# @return [true, false, nil]
|
58
|
+
def shutdown?
|
59
|
+
!@timer&.running?
|
60
|
+
end
|
61
|
+
|
62
|
+
# Restart the poller.
|
63
|
+
# When shutdown, start; or shutdown and start.
|
64
|
+
# @param wait [Boolean] Wait for background thread to finish
|
65
|
+
# @return [void]
|
66
|
+
def restart(wait: true)
|
67
|
+
shutdown(wait: wait)
|
68
|
+
create_pool
|
69
|
+
end
|
70
|
+
|
71
|
+
# Invoked on completion of TimerTask task.
|
72
|
+
# @!visibility private
|
73
|
+
# @return [void]
|
74
|
+
def timer_observer(time, executed_task, thread_error)
|
75
|
+
GoodJob.on_thread_error.call(thread_error) if thread_error && GoodJob.on_thread_error.respond_to?(:call)
|
76
|
+
instrument("finished_timer_task", { result: executed_task, error: thread_error, time: time })
|
77
|
+
end
|
78
|
+
|
79
|
+
private
|
80
|
+
|
81
|
+
def create_pool
|
82
|
+
return if @timer_options[:execution_interval] <= 0
|
83
|
+
|
84
|
+
@timer = Concurrent::TimerTask.new(@timer_options) do
|
85
|
+
recipients.each do |recipient|
|
86
|
+
target, method_name = recipient.is_a?(Array) ? recipient : [recipient, :call]
|
87
|
+
target.send(method_name)
|
88
|
+
end
|
89
|
+
end
|
90
|
+
@timer.add_observer(self, :timer_observer)
|
91
|
+
@timer.execute
|
92
|
+
end
|
93
|
+
end
|
94
|
+
end
|
data/lib/good_job/scheduler.rb
CHANGED
@@ -14,20 +14,12 @@ module GoodJob # :nodoc:
|
|
14
14
|
# The scheduler maintains an instance of +Concurrent::TimerTask+, which wakes sleeping threads and causes them to check whether the performer has new work.
|
15
15
|
#
|
16
16
|
class Scheduler
|
17
|
-
# Defaults for instance of Concurrent::TimerTask.
|
18
|
-
# The timer controls how and when sleeping threads check for new work.
|
19
|
-
DEFAULT_TIMER_OPTIONS = {
|
20
|
-
execution_interval: 1,
|
21
|
-
timeout_interval: 1,
|
22
|
-
run_now: true,
|
23
|
-
}.freeze
|
24
|
-
|
25
17
|
# Defaults for instance of Concurrent::ThreadPoolExecutor
|
26
18
|
# The thread pool is where work is performed.
|
27
19
|
DEFAULT_POOL_OPTIONS = {
|
28
20
|
name: name,
|
29
21
|
min_threads: 0,
|
30
|
-
max_threads:
|
22
|
+
max_threads: Configuration::DEFAULT_MAX_THREADS,
|
31
23
|
auto_terminate: true,
|
32
24
|
idletime: 60,
|
33
25
|
max_queue: -1,
|
@@ -41,7 +33,6 @@ module GoodJob # :nodoc:
|
|
41
33
|
cattr_reader :instances, default: [], instance_reader: false
|
42
34
|
|
43
35
|
# Creates GoodJob::Scheduler(s) and Performers from a GoodJob::Configuration instance.
|
44
|
-
# TODO: move this to GoodJob::Configuration
|
45
36
|
# @param configuration [GoodJob::Configuration]
|
46
37
|
# @return [GoodJob::Scheduler, GoodJob::MultiScheduler]
|
47
38
|
def self.from_configuration(configuration)
|
@@ -53,7 +44,7 @@ module GoodJob # :nodoc:
|
|
53
44
|
parsed = GoodJob::Job.queue_parser(queue_string)
|
54
45
|
job_filter = proc do |state|
|
55
46
|
if parsed[:exclude]
|
56
|
-
|
47
|
+
parsed[:exclude].exclude?(state[:queue_name])
|
57
48
|
elsif parsed[:include]
|
58
49
|
parsed[:include].include? state[:queue_name]
|
59
50
|
else
|
@@ -62,14 +53,7 @@ module GoodJob # :nodoc:
|
|
62
53
|
end
|
63
54
|
job_performer = GoodJob::Performer.new(job_query, :perform_with_advisory_lock, name: queue_string, filter: job_filter)
|
64
55
|
|
65
|
-
|
66
|
-
timer_options[:execution_interval] = configuration.poll_interval
|
67
|
-
|
68
|
-
pool_options = {
|
69
|
-
max_threads: max_threads,
|
70
|
-
}
|
71
|
-
|
72
|
-
GoodJob::Scheduler.new(job_performer, timer_options: timer_options, pool_options: pool_options)
|
56
|
+
GoodJob::Scheduler.new(job_performer, max_threads: max_threads)
|
73
57
|
end
|
74
58
|
|
75
59
|
if schedulers.size > 1
|
@@ -80,21 +64,19 @@ module GoodJob # :nodoc:
|
|
80
64
|
end
|
81
65
|
|
82
66
|
# @param performer [GoodJob::Performer]
|
83
|
-
# @param
|
84
|
-
|
85
|
-
def initialize(performer, timer_options: {}, pool_options: {})
|
86
|
-
# TODO: Replace `timer_options` and `pool_options` with only `poll_interval` and `max_threads`
|
67
|
+
# @param max_threads [Numeric, nil] number of seconds between polls for jobs
|
68
|
+
def initialize(performer, max_threads: nil)
|
87
69
|
raise ArgumentError, "Performer argument must implement #next" unless performer.respond_to?(:next)
|
88
70
|
|
89
71
|
self.class.instances << self
|
90
72
|
|
91
73
|
@performer = performer
|
92
|
-
@pool_options = DEFAULT_POOL_OPTIONS.merge(pool_options)
|
93
|
-
@timer_options = DEFAULT_TIMER_OPTIONS.merge(timer_options)
|
94
74
|
|
95
|
-
@pool_options
|
75
|
+
@pool_options = DEFAULT_POOL_OPTIONS.dup
|
76
|
+
@pool_options[:max_threads] = max_threads if max_threads.present?
|
77
|
+
@pool_options[:name] = "GoodJob::Scheduler(queues=#{@performer.name} max_threads=#{@pool_options[:max_threads]})"
|
96
78
|
|
97
|
-
|
79
|
+
create_pool
|
98
80
|
end
|
99
81
|
|
100
82
|
# Shut down the scheduler.
|
@@ -105,28 +87,20 @@ module GoodJob # :nodoc:
|
|
105
87
|
# @param wait [Boolean] Wait for actively executing jobs to finish
|
106
88
|
# @return [void]
|
107
89
|
def shutdown(wait: true)
|
108
|
-
|
90
|
+
return unless @pool&.running?
|
109
91
|
|
110
92
|
instrument("scheduler_shutdown_start", { wait: wait })
|
111
93
|
instrument("scheduler_shutdown", { wait: wait }) do
|
112
|
-
|
113
|
-
|
114
|
-
|
115
|
-
# TODO: Should be killed if wait is not true
|
116
|
-
end
|
117
|
-
|
118
|
-
if @pool&.running?
|
119
|
-
@pool.shutdown
|
120
|
-
@pool.wait_for_termination if wait
|
121
|
-
# TODO: Should be killed if wait is not true
|
122
|
-
end
|
94
|
+
@pool.shutdown
|
95
|
+
@pool.wait_for_termination if wait
|
96
|
+
# TODO: Should be killed if wait is not true
|
123
97
|
end
|
124
98
|
end
|
125
99
|
|
126
100
|
# Tests whether the scheduler is shutdown.
|
127
101
|
# @return [true, false, nil]
|
128
102
|
def shutdown?
|
129
|
-
|
103
|
+
!@pool&.running?
|
130
104
|
end
|
131
105
|
|
132
106
|
# Restart the Scheduler.
|
@@ -136,8 +110,7 @@ module GoodJob # :nodoc:
|
|
136
110
|
def restart(wait: true)
|
137
111
|
instrument("scheduler_restart_pools") do
|
138
112
|
shutdown(wait: wait) unless shutdown?
|
139
|
-
|
140
|
-
@_shutdown = false
|
113
|
+
create_pool
|
141
114
|
end
|
142
115
|
end
|
143
116
|
|
@@ -162,14 +135,6 @@ module GoodJob # :nodoc:
|
|
162
135
|
true
|
163
136
|
end
|
164
137
|
|
165
|
-
# Invoked on completion of TimerTask task.
|
166
|
-
# @!visibility private
|
167
|
-
# @return [void]
|
168
|
-
def timer_observer(time, executed_task, thread_error)
|
169
|
-
GoodJob.on_thread_error.call(thread_error) if thread_error && GoodJob.on_thread_error.respond_to?(:call)
|
170
|
-
instrument("finished_timer_task", { result: executed_task, error: thread_error, time: time })
|
171
|
-
end
|
172
|
-
|
173
138
|
# Invoked on completion of ThreadPoolExecutor task
|
174
139
|
# @!visibility private
|
175
140
|
# @return [void]
|
@@ -181,14 +146,9 @@ module GoodJob # :nodoc:
|
|
181
146
|
|
182
147
|
private
|
183
148
|
|
184
|
-
def
|
185
|
-
instrument("
|
149
|
+
def create_pool
|
150
|
+
instrument("scheduler_create_pool", { performer_name: @performer.name, max_threads: @pool_options[:max_threads] }) do
|
186
151
|
@pool = ThreadPoolExecutor.new(@pool_options)
|
187
|
-
next unless @timer_options[:execution_interval].positive?
|
188
|
-
|
189
|
-
@timer = Concurrent::TimerTask.new(@timer_options) { create_thread }
|
190
|
-
@timer.add_observer(self, :timer_observer)
|
191
|
-
@timer.execute
|
192
152
|
end
|
193
153
|
end
|
194
154
|
|
@@ -201,20 +161,20 @@ module GoodJob # :nodoc:
|
|
201
161
|
|
202
162
|
ActiveSupport::Notifications.instrument("#{name}.good_job", payload, &block)
|
203
163
|
end
|
204
|
-
end
|
205
164
|
|
206
|
-
|
207
|
-
|
208
|
-
|
209
|
-
|
210
|
-
|
211
|
-
|
212
|
-
|
213
|
-
|
214
|
-
|
215
|
-
|
216
|
-
|
217
|
-
|
165
|
+
# Custom sub-class of +Concurrent::ThreadPoolExecutor+ to add additional worker status.
|
166
|
+
# @private
|
167
|
+
class ThreadPoolExecutor < Concurrent::ThreadPoolExecutor
|
168
|
+
# Number of inactive threads available to execute tasks.
|
169
|
+
# https://github.com/ruby-concurrency/concurrent-ruby/issues/684#issuecomment-427594437
|
170
|
+
# @return [Integer]
|
171
|
+
def ready_worker_count
|
172
|
+
synchronize do
|
173
|
+
workers_still_to_be_created = @max_length - @pool.length
|
174
|
+
workers_created_but_waiting = @ready.length
|
175
|
+
|
176
|
+
workers_still_to_be_created + workers_created_but_waiting
|
177
|
+
end
|
218
178
|
end
|
219
179
|
end
|
220
180
|
end
|
data/lib/good_job/version.rb
CHANGED
metadata
CHANGED
@@ -1,14 +1,14 @@
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
2
2
|
name: good_job
|
3
3
|
version: !ruby/object:Gem::Version
|
4
|
-
version: 1.
|
4
|
+
version: 1.3.3
|
5
5
|
platform: ruby
|
6
6
|
authors:
|
7
7
|
- Ben Sheldon
|
8
8
|
autorequire:
|
9
9
|
bindir: exe
|
10
10
|
cert_chain: []
|
11
|
-
date: 2020-
|
11
|
+
date: 2020-12-01 00:00:00.000000000 Z
|
12
12
|
dependencies:
|
13
13
|
- !ruby/object:Gem::Dependency
|
14
14
|
name: activejob
|
@@ -290,6 +290,20 @@ dependencies:
|
|
290
290
|
- - ">="
|
291
291
|
- !ruby/object:Gem::Version
|
292
292
|
version: '0'
|
293
|
+
- !ruby/object:Gem::Dependency
|
294
|
+
name: rails
|
295
|
+
requirement: !ruby/object:Gem::Requirement
|
296
|
+
requirements:
|
297
|
+
- - ">="
|
298
|
+
- !ruby/object:Gem::Version
|
299
|
+
version: '0'
|
300
|
+
type: :development
|
301
|
+
prerelease: false
|
302
|
+
version_requirements: !ruby/object:Gem::Requirement
|
303
|
+
requirements:
|
304
|
+
- - ">="
|
305
|
+
- !ruby/object:Gem::Version
|
306
|
+
version: '0'
|
293
307
|
- !ruby/object:Gem::Dependency
|
294
308
|
name: rbtrace
|
295
309
|
requirement: !ruby/object:Gem::Requirement
|
@@ -475,6 +489,7 @@ files:
|
|
475
489
|
- lib/good_job/multi_scheduler.rb
|
476
490
|
- lib/good_job/notifier.rb
|
477
491
|
- lib/good_job/performer.rb
|
492
|
+
- lib/good_job/poller.rb
|
478
493
|
- lib/good_job/railtie.rb
|
479
494
|
- lib/good_job/scheduler.rb
|
480
495
|
- lib/good_job/version.rb
|
@@ -510,7 +525,7 @@ required_rubygems_version: !ruby/object:Gem::Requirement
|
|
510
525
|
- !ruby/object:Gem::Version
|
511
526
|
version: '0'
|
512
527
|
requirements: []
|
513
|
-
rubygems_version: 3.
|
528
|
+
rubygems_version: 3.1.4
|
514
529
|
signing_key:
|
515
530
|
specification_version: 4
|
516
531
|
summary: A multithreaded, Postgres-based ActiveJob backend for Ruby on Rails
|