good_job 1.0.0 → 1.1.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/CHANGELOG.md +109 -4
- data/README.md +223 -56
- data/lib/active_job/queue_adapters/good_job_adapter.rb +3 -12
- data/lib/generators/good_job/install_generator.rb +24 -0
- data/lib/generators/good_job/templates/migration.rb.erb +20 -0
- data/lib/good_job.rb +6 -2
- data/lib/good_job/adapter.rb +30 -16
- data/lib/good_job/cli.rb +18 -48
- data/lib/good_job/configuration.rb +55 -0
- data/lib/good_job/job.rb +30 -25
- data/lib/good_job/log_subscriber.rb +110 -0
- data/lib/good_job/multi_scheduler.rb +25 -0
- data/lib/good_job/performer.rb +4 -1
- data/lib/good_job/railtie.rb +1 -0
- data/lib/good_job/scheduler.rb +83 -16
- data/lib/good_job/version.rb +1 -1
- metadata +38 -6
- data/lib/good_job/logging.rb +0 -70
checksums.yaml
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
---
|
2
2
|
SHA256:
|
3
|
-
metadata.gz:
|
4
|
-
data.tar.gz:
|
3
|
+
metadata.gz: b7b7100f183bea75e74273f8c60b87eb00f234b56196f1239ad5c1677e216250
|
4
|
+
data.tar.gz: b22120c6cdb91f38e201ef2fd8dd150f7775462ba27e7b9646bad504384c94d0
|
5
5
|
SHA512:
|
6
|
-
metadata.gz:
|
7
|
-
data.tar.gz:
|
6
|
+
metadata.gz: '0844d72f1fb34e1608f40d6c134a3a24e8e5e0876ca4c96eaa246568db87f64b47be8170e5061517f455148da5e40fdd0a278a8a790a93a7ddb64eb024fdb35f'
|
7
|
+
data.tar.gz: 20934ee8ab6fc277519c2a551ee44d6803d09405215faff35fda4227e88729d720968235003e694f83887cea82709caa68eabc7afbb10335b2a671b3bddfe604
|
data/CHANGELOG.md
CHANGED
@@ -1,24 +1,130 @@
|
|
1
1
|
# Changelog
|
2
2
|
|
3
|
+
## [v1.1.1](https://github.com/bensheldon/good_job/tree/v1.1.1) (2020-08-12)
|
4
|
+
|
5
|
+
[Full Changelog](https://github.com/bensheldon/good_job/compare/v1.1.0...v1.1.1)
|
6
|
+
|
7
|
+
**Implemented enhancements:**
|
8
|
+
|
9
|
+
- Allow multiple schedulers within the same process. e.g. `queues=mice:2,elephants:4` [\#45](https://github.com/bensheldon/good_job/issues/45)
|
10
|
+
|
11
|
+
**Merged pull requests:**
|
12
|
+
|
13
|
+
- Allow instantiation of multiple schedulers via --queues [\#76](https://github.com/bensheldon/good_job/pull/76) ([bensheldon](https://github.com/bensheldon))
|
14
|
+
- Extract options parsing to Configuration object [\#74](https://github.com/bensheldon/good_job/pull/74) ([bensheldon](https://github.com/bensheldon))
|
15
|
+
|
16
|
+
## [v1.1.0](https://github.com/bensheldon/good_job/tree/v1.1.0) (2020-08-10)
|
17
|
+
|
18
|
+
[Full Changelog](https://github.com/bensheldon/good_job/compare/v1.0.3...v1.1.0)
|
19
|
+
|
20
|
+
**Closed issues:**
|
21
|
+
|
22
|
+
- Document reliability guarantees [\#59](https://github.com/bensheldon/good_job/issues/59)
|
23
|
+
- Document how to hook in exception monitor \(Sentry, Rollbar, etc\) [\#47](https://github.com/bensheldon/good_job/issues/47)
|
24
|
+
- Allow an Async mode [\#27](https://github.com/bensheldon/good_job/issues/27)
|
25
|
+
|
26
|
+
**Merged pull requests:**
|
27
|
+
|
28
|
+
- Add a callable hook on thread errors [\#71](https://github.com/bensheldon/good_job/pull/71) ([bensheldon](https://github.com/bensheldon))
|
29
|
+
- Clarify reliability guarantees [\#70](https://github.com/bensheldon/good_job/pull/70) ([bensheldon](https://github.com/bensheldon))
|
30
|
+
- Clean up Readme formatting; re-arrange tests for clarity and values [\#69](https://github.com/bensheldon/good_job/pull/69) ([bensheldon](https://github.com/bensheldon))
|
31
|
+
- Create an Async execution mode [\#68](https://github.com/bensheldon/good_job/pull/68) ([bensheldon](https://github.com/bensheldon))
|
32
|
+
- Move all stdout to LogSubscriber [\#67](https://github.com/bensheldon/good_job/pull/67) ([bensheldon](https://github.com/bensheldon))
|
33
|
+
- Allow schedulers to be restarted; separate unit tests from integration tests [\#66](https://github.com/bensheldon/good_job/pull/66) ([bensheldon](https://github.com/bensheldon))
|
34
|
+
|
35
|
+
## [v1.0.3](https://github.com/bensheldon/good_job/tree/v1.0.3) (2020-07-26)
|
36
|
+
|
37
|
+
[Full Changelog](https://github.com/bensheldon/good_job/compare/v1.0.2...v1.0.3)
|
38
|
+
|
39
|
+
**Fixed bugs:**
|
40
|
+
|
41
|
+
- Preserve GoodJob::Jobs when a StandardError is raised [\#60](https://github.com/bensheldon/good_job/issues/60)
|
42
|
+
|
43
|
+
**Closed issues:**
|
44
|
+
|
45
|
+
- Have an initial setup generator [\#6](https://github.com/bensheldon/good_job/issues/6)
|
46
|
+
|
47
|
+
**Merged pull requests:**
|
48
|
+
|
49
|
+
- Re-perform a job if a StandardError bubbles up; better document job reliability [\#62](https://github.com/bensheldon/good_job/pull/62) ([bensheldon](https://github.com/bensheldon))
|
50
|
+
- Update the setup documentation to use correct bin setup command [\#61](https://github.com/bensheldon/good_job/pull/61) ([jm96441n](https://github.com/jm96441n))
|
51
|
+
|
52
|
+
## [v1.0.2](https://github.com/bensheldon/good_job/tree/v1.0.2) (2020-07-25)
|
53
|
+
|
54
|
+
[Full Changelog](https://github.com/bensheldon/good_job/compare/v1.0.1...v1.0.2)
|
55
|
+
|
56
|
+
**Fixed bugs:**
|
57
|
+
|
58
|
+
- Fix counting of available execution threads [\#58](https://github.com/bensheldon/good_job/pull/58) ([bensheldon](https://github.com/bensheldon))
|
59
|
+
|
60
|
+
**Merged pull requests:**
|
61
|
+
|
62
|
+
- Add migration generator [\#56](https://github.com/bensheldon/good_job/pull/56) ([thedanbob](https://github.com/thedanbob))
|
63
|
+
- Fix migration script in readme [\#55](https://github.com/bensheldon/good_job/pull/55) ([thedanbob](https://github.com/thedanbob))
|
64
|
+
|
65
|
+
## [v1.0.1](https://github.com/bensheldon/good_job/tree/v1.0.1) (2020-07-22)
|
66
|
+
|
67
|
+
[Full Changelog](https://github.com/bensheldon/good_job/compare/v1.0.0...v1.0.1)
|
68
|
+
|
69
|
+
**Merged pull requests:**
|
70
|
+
|
71
|
+
- Change threadpool idletime default to 60 seconds from 0 [\#49](https://github.com/bensheldon/good_job/pull/49) ([bensheldon](https://github.com/bensheldon))
|
72
|
+
|
3
73
|
## [v1.0.0](https://github.com/bensheldon/good_job/tree/v1.0.0) (2020-07-20)
|
4
74
|
|
5
|
-
[Full Changelog](https://github.com/bensheldon/good_job/compare/v0.
|
75
|
+
[Full Changelog](https://github.com/bensheldon/good_job/compare/v0.9.0...v1.0.0)
|
76
|
+
|
77
|
+
## [v0.9.0](https://github.com/bensheldon/good_job/tree/v0.9.0) (2020-07-20)
|
78
|
+
|
79
|
+
[Full Changelog](https://github.com/bensheldon/good_job/compare/v0.8.2...v0.9.0)
|
80
|
+
|
81
|
+
**Merged pull requests:**
|
82
|
+
|
83
|
+
- Allow preservation of finished job records [\#46](https://github.com/bensheldon/good_job/pull/46) ([bensheldon](https://github.com/bensheldon))
|
84
|
+
|
85
|
+
## [v0.8.2](https://github.com/bensheldon/good_job/tree/v0.8.2) (2020-07-18)
|
86
|
+
|
87
|
+
[Full Changelog](https://github.com/bensheldon/good_job/compare/v0.8.1...v0.8.2)
|
6
88
|
|
7
89
|
**Closed issues:**
|
8
90
|
|
9
|
-
- Always store a default priority \(0\) and scheduled\_at\(Time.current\) [\#30](https://github.com/bensheldon/good_job/issues/30)
|
10
91
|
- Add a job timeout configuration to time out jobs that have run too long [\#19](https://github.com/bensheldon/good_job/issues/19)
|
11
92
|
|
12
93
|
**Merged pull requests:**
|
13
94
|
|
14
|
-
- Allow preservation of finished job records [\#46](https://github.com/bensheldon/good_job/pull/46) ([bensheldon](https://github.com/bensheldon))
|
15
95
|
- Run Github Action tests on PRs from forks [\#44](https://github.com/bensheldon/good_job/pull/44) ([bensheldon](https://github.com/bensheldon))
|
16
96
|
- Fix Rubygems homepage URL [\#43](https://github.com/bensheldon/good_job/pull/43) ([joshmn](https://github.com/joshmn))
|
97
|
+
|
98
|
+
## [v0.8.1](https://github.com/bensheldon/good_job/tree/v0.8.1) (2020-07-18)
|
99
|
+
|
100
|
+
[Full Changelog](https://github.com/bensheldon/good_job/compare/v0.8.0...v0.8.1)
|
101
|
+
|
102
|
+
**Merged pull requests:**
|
103
|
+
|
17
104
|
- Move where\(scheduled\_at: Time.current\) into dynamic part of GoodJob::Job::Performer [\#42](https://github.com/bensheldon/good_job/pull/42) ([bensheldon](https://github.com/bensheldon))
|
105
|
+
|
106
|
+
## [v0.8.0](https://github.com/bensheldon/good_job/tree/v0.8.0) (2020-07-17)
|
107
|
+
|
108
|
+
[Full Changelog](https://github.com/bensheldon/good_job/compare/v0.7.0...v0.8.0)
|
109
|
+
|
110
|
+
**Merged pull requests:**
|
111
|
+
|
18
112
|
- Replace Adapter inline boolean kwarg with execution\_mode instead [\#41](https://github.com/bensheldon/good_job/pull/41) ([bensheldon](https://github.com/bensheldon))
|
113
|
+
|
114
|
+
## [v0.7.0](https://github.com/bensheldon/good_job/tree/v0.7.0) (2020-07-16)
|
115
|
+
|
116
|
+
[Full Changelog](https://github.com/bensheldon/good_job/compare/v0.6.0...v0.7.0)
|
117
|
+
|
118
|
+
**Closed issues:**
|
119
|
+
|
120
|
+
- Always store a default priority \(0\) and scheduled\_at\(Time.current\) [\#30](https://github.com/bensheldon/good_job/issues/30)
|
121
|
+
|
122
|
+
**Merged pull requests:**
|
123
|
+
|
19
124
|
- Add more examples to Readme [\#39](https://github.com/bensheldon/good_job/pull/39) ([bensheldon](https://github.com/bensheldon))
|
20
125
|
- Add additional Rubocops and lint [\#38](https://github.com/bensheldon/good_job/pull/38) ([bensheldon](https://github.com/bensheldon))
|
21
126
|
- Always store a default queue\_name, priority and scheduled\_at; index by queue\_name and scheduled\_at [\#37](https://github.com/bensheldon/good_job/pull/37) ([bensheldon](https://github.com/bensheldon))
|
127
|
+
- Extract Job querying behavior out of Scheduler [\#31](https://github.com/bensheldon/good_job/pull/31) ([bensheldon](https://github.com/bensheldon))
|
22
128
|
|
23
129
|
## [v0.6.0](https://github.com/bensheldon/good_job/tree/v0.6.0) (2020-07-15)
|
24
130
|
|
@@ -34,7 +140,6 @@
|
|
34
140
|
- Improve generation of changelog [\#36](https://github.com/bensheldon/good_job/pull/36) ([bensheldon](https://github.com/bensheldon))
|
35
141
|
- Update Github Action Workflow for Backlog Project Board [\#35](https://github.com/bensheldon/good_job/pull/35) ([bensheldon](https://github.com/bensheldon))
|
36
142
|
- Add configuration options to good\_job executable [\#33](https://github.com/bensheldon/good_job/pull/33) ([bensheldon](https://github.com/bensheldon))
|
37
|
-
- Extract Job querying behavior out of Scheduler [\#31](https://github.com/bensheldon/good_job/pull/31) ([bensheldon](https://github.com/bensheldon))
|
38
143
|
- Allow configuration of Rails queue adapter with `:good\_job` [\#28](https://github.com/bensheldon/good_job/pull/28) ([bensheldon](https://github.com/bensheldon))
|
39
144
|
|
40
145
|
## [v0.5.0](https://github.com/bensheldon/good_job/tree/v0.5.0) (2020-07-13)
|
data/README.md
CHANGED
@@ -9,6 +9,8 @@ GoodJob is a multithreaded, Postgres-based, ActiveJob backend for Ruby on Rails.
|
|
9
9
|
- **Backed by Postgres.** Relies upon Postgres integrity and session-level Advisory Locks to provide run-once safety and stay within the limits of `schema.rb`.
|
10
10
|
- **For most workloads.** Targets full-stack teams, economy-minded solo developers, and applications that enqueue less than 1-million jobs/day.
|
11
11
|
|
12
|
+
For more of the story of GoodJob, read the [introductory blog post](https://island94.org/2020/07/introducing-goodjob-1-0).
|
13
|
+
|
12
14
|
## Installation
|
13
15
|
|
14
16
|
Add this line to your application's Gemfile:
|
@@ -25,35 +27,11 @@ $ bundle install
|
|
25
27
|
## Usage
|
26
28
|
|
27
29
|
1. Create a database migration:
|
28
|
-
|
29
|
-
|
30
|
+
|
31
|
+
```bash
|
32
|
+
$ bin/rails g good_job:install
|
30
33
|
```
|
31
34
|
|
32
|
-
Add to the newly created migration file:
|
33
|
-
|
34
|
-
```ruby
|
35
|
-
class CreateGoodJobs < ActiveRecord::Migration[6.0]
|
36
|
-
def change
|
37
|
-
enable_extension 'pgcrypto'
|
38
|
-
|
39
|
-
create_table :good_jobs, id: :uuid do |t|
|
40
|
-
t.timestamps
|
41
|
-
|
42
|
-
t.text :queue_name
|
43
|
-
t.integer :priority
|
44
|
-
t.jsonb :serialized_params
|
45
|
-
t.timestamp :scheduled_at
|
46
|
-
t.timestamp :performed_at
|
47
|
-
t.timestamp :finished_at
|
48
|
-
t.text :error
|
49
|
-
|
50
|
-
add_index :good_jobs, :scheduled_at, where: "(finished_at IS NULL)"
|
51
|
-
add_index :good_jobs, [:queue_name, :scheduled_at], where: "(finished_at IS NULL)"
|
52
|
-
end
|
53
|
-
end
|
54
|
-
end
|
55
|
-
```
|
56
|
-
|
57
35
|
Run the migration:
|
58
36
|
|
59
37
|
```bash
|
@@ -61,7 +39,8 @@ $ bundle install
|
|
61
39
|
```
|
62
40
|
|
63
41
|
1. Configure the ActiveJob adapter:
|
64
|
-
|
42
|
+
|
43
|
+
```ruby
|
65
44
|
# config/application.rb
|
66
45
|
config.active_job.queue_adapter = :good_job
|
67
46
|
```
|
@@ -80,48 +59,183 @@ $ bundle install
|
|
80
59
|
```
|
81
60
|
|
82
61
|
1. Queue your job 🎉:
|
62
|
+
|
83
63
|
```ruby
|
84
64
|
YourJob.set(queue: :some_queue, wait: 5.minutes, priority: 10).perform_later
|
85
65
|
```
|
86
66
|
|
87
67
|
1. In production, the scheduler is designed to run in its own process:
|
68
|
+
|
88
69
|
```bash
|
89
70
|
$ bundle exec good_job
|
90
71
|
```
|
91
72
|
|
92
73
|
Configuration options available with `help`:
|
93
|
-
|
94
|
-
|
74
|
+
|
75
|
+
```bash
|
76
|
+
$ bundle exec good_job help start
|
77
|
+
|
78
|
+
Usage:
|
79
|
+
good_job start
|
80
|
+
|
81
|
+
Options:
|
82
|
+
[--max-threads=N] # Maximum number of threads to use for working jobs (default: ActiveRecord::Base.connection_pool.size)
|
83
|
+
[--queues=queue1,queue2(;queue3,queue4:5)] # Queues to work from. Separate multiple queues with commas; separate isolated execution pools with semicolons and threads with colons (default: *)
|
84
|
+
[--poll-interval=N] # Interval between polls for available jobs in seconds (default: 1)
|
85
|
+
|
86
|
+
Start job worker
|
87
|
+
```
|
88
|
+
|
89
|
+
1. Optimize execution to reduce congestion and execution latency. By default, GoodJob creates a single thread execution pool that will execute jobs from any queue. Depending on your application's workload, job types, and service level objectives, you may wish to optimize execution resources; for example, providing dedicated execution resources for transactional emails so they are not delayed by long-running batch jobs. Some options:
|
90
|
+
|
91
|
+
- Multiple execution pools within a single process:
|
92
|
+
|
93
|
+
```bash
|
94
|
+
$ bundle exec good_job --queues=*;transactional_messages:2;batch_processing:1 --max-threads=5
|
95
|
+
```
|
96
|
+
|
97
|
+
This configuration will result in a single process with 3 isolated thread execution pools. A pool that will run jobs from any queue, `*`, with up to 5 threads; a pool that will only run jobs enqueued on `transactional_messages` with up to 2 threads; and a pool dedicated to the `batch_processing` queue with a single thread.
|
98
|
+
|
99
|
+
For moderate workloads, multiple isolated thread execution pools offers a good balance between congestion management and economy.
|
100
|
+
|
101
|
+
Configuration can be injected by environment variables too:
|
102
|
+
|
103
|
+
```bash
|
104
|
+
$ GOOD_JOB_QUEUES="*;transactional_messages:2;batch_processing:1" GOOD_JOB_MAX_THREADS=5 bundle exec good_job
|
105
|
+
```
|
106
|
+
|
107
|
+
- Multiple processes; for example, on Heroku:
|
108
|
+
|
109
|
+
```procfile
|
110
|
+
# Procfile
|
111
|
+
|
112
|
+
# Separate dyno types
|
113
|
+
worker: bundle exec good_job --max-threads=5
|
114
|
+
transactional_worker: bundle exec good_job --queues=transactional_messages --max-threads=2
|
115
|
+
batch_worker: bundle exec good_job --queues=batch_processing --max-threads=1
|
116
|
+
|
117
|
+
# Combined multi-process dyno
|
118
|
+
combined_worker: bundle exec good_job --max-threads=5 & bundle exec good_job --queues=transactional_messages --max-threads=2 & bundle exec good_job --queues=batch_processing --max-threads=1 & wait -n
|
119
|
+
```
|
120
|
+
|
121
|
+
Running multiple processes can optimize for CPU performance at the expense of greater memory and system resource usage.
|
122
|
+
|
123
|
+
_Keep in mind, queue operations and management is an advanced discipline. This stuff is complex, especially for heavy workloads and unique processing requirements. Good job 👍_
|
124
|
+
|
125
|
+
### Error handling, retries, and reliability
|
126
|
+
|
127
|
+
GoodJob guarantees that a completely-performed job will run once and only once. GoodJob fully supports ActiveJob's built-in functionality for error handling, retries and timeouts. Writing reliable, transactional, and idempotent `ActiveJob#perform` methods is outside the scope of GoodJob.
|
128
|
+
|
129
|
+
#### Error handling
|
130
|
+
|
131
|
+
By default, if a job raises an error while it is being performed, _and it bubbles up to the GoodJob backend_, GoodJob will be immediately re-perform the job until it finishes successfully.
|
132
|
+
|
133
|
+
- `Exception`-type errors, such as a SIGINT, will always cause a job to be re-performed.
|
134
|
+
- `StandardError`-type errors, by default, will cause a job to be re-performed, though this is configurable:
|
95
135
|
|
96
|
-
|
97
|
-
|
98
|
-
|
99
|
-
|
100
|
-
|
101
|
-
|
102
|
-
|
103
|
-
|
136
|
+
```ruby
|
137
|
+
# config/initializers/good_job.rb
|
138
|
+
GoodJob.reperform_jobs_on_standard_error = true # => default
|
139
|
+
```
|
140
|
+
|
141
|
+
To report errors that _do_ bubble up to the GoodJob backend, assign a callable to `GoodJob.on_thread_error`. For example:
|
142
|
+
|
143
|
+
```ruby
|
144
|
+
# config/initializers/good_job.rb
|
145
|
+
|
146
|
+
# With Sentry (or Bugsnag, Airbrake, Honeybadger, etc.)
|
147
|
+
GoodJob.on_thread_error = -> (exception) { Raven.capture_exception(exception) }
|
148
|
+
```
|
149
|
+
|
150
|
+
### Retrying jobs
|
151
|
+
|
152
|
+
ActiveJob can be configured to retry an infinite number of times, with an exponential backoff. Using ActiveJob's `retry_on` will ensure that errors do not bubble up to the GoodJob backend:
|
153
|
+
|
154
|
+
```ruby
|
155
|
+
class ApplicationJob < ActiveJob::Base
|
156
|
+
retry_on StandardError, wait: :exponentially_longer, attempts: Float::INFINITY
|
157
|
+
# ...
|
158
|
+
end
|
159
|
+
```
|
160
|
+
|
161
|
+
When specifying a limited number of retries, care must be taken to ensure that an error does not bubble up to the GoodJob backend because that will result in the job being re-performed:
|
162
|
+
|
163
|
+
```ruby
|
164
|
+
class ApplicationJob < ActiveJob::Base
|
165
|
+
retry_on StandardError, attempts: 5 do |_job, _exception|
|
166
|
+
# Log error, etc.
|
167
|
+
# You must implement this block, otherwise,
|
168
|
+
# Active Job will re-raise the error.
|
169
|
+
# Do not re-raise the error, otherwise
|
170
|
+
# GoodJob will immediately re-perform the job.
|
171
|
+
end
|
172
|
+
# ...
|
173
|
+
end
|
174
|
+
```
|
104
175
|
|
105
|
-
|
176
|
+
GoodJob can be configured to allow omitting `retry_on`'s block argument and implicitly discard un-handled errors:
|
177
|
+
|
178
|
+
```ruby
|
179
|
+
# config/initializers/good_job.rb
|
180
|
+
|
181
|
+
# Do NOT re-perform a job if a StandardError bubbles up to the GoodJob backend
|
182
|
+
GoodJob.reperform_jobs_on_standard_error = false
|
183
|
+
```
|
106
184
|
|
107
|
-
|
185
|
+
When using an exception monitoring service (e.g. Sentry, Bugsnag, Airbrake, Honeybadger, etc), the use of `rescue_on` may be incompatible with their ActiveJob integration. It's safest to explicitly wrap jobs with an exception reporter. For example:
|
108
186
|
|
109
187
|
```ruby
|
110
188
|
class ApplicationJob < ActiveJob::Base
|
111
|
-
# Retry errors an infinite number of times with exponential back-off
|
112
189
|
retry_on StandardError, wait: :exponentially_longer, attempts: Float::INFINITY
|
190
|
+
|
191
|
+
around_perform do |_job, block|
|
192
|
+
block.call
|
193
|
+
rescue StandardError => e
|
194
|
+
Raven.capture_exception(e)
|
195
|
+
raise
|
196
|
+
end
|
197
|
+
# ...
|
198
|
+
end
|
199
|
+
```
|
200
|
+
|
201
|
+
|
202
|
+
ActiveJob's `discard_on` functionality is supported too.
|
203
|
+
|
204
|
+
#### ActionMailer retries
|
205
|
+
|
206
|
+
Using a Mailer's `#deliver_later` will enqueue an instance of `ActionMailer::DeliveryJob` which inherits from `ActiveJob::Base` rather than your applications `ApplicationJob`. You can use an initializer to configure retries on `ActionMailer::DeliveryJob`:
|
207
|
+
|
208
|
+
```ruby
|
209
|
+
# config/initializers/good_job.rb
|
210
|
+
ActionMailer::DeliveryJob.retry_on StandardError, wait: :exponentially_longer, attempts: Float::INFINITY
|
211
|
+
|
212
|
+
# With Sentry (or Bugsnag, Airbrake, Honeybadger, etc.)
|
213
|
+
ActionMailer::DeliveryJob.around_perform do |_job, block|
|
214
|
+
block.call
|
215
|
+
rescue StandardError => e
|
216
|
+
Raven.capture_exception(e)
|
217
|
+
raise
|
218
|
+
end
|
219
|
+
```
|
113
220
|
|
114
|
-
|
221
|
+
#### Timeouts
|
222
|
+
|
223
|
+
Job timeouts can be configured with an `around_perform`:
|
224
|
+
|
225
|
+
```ruby
|
226
|
+
class ApplicationJob < ActiveJob::Base
|
115
227
|
JobTimeoutError = Class.new(StandardError)
|
228
|
+
|
116
229
|
around_perform do |_job, block|
|
230
|
+
# Timeout jobs after 10 minutes
|
117
231
|
Timeout.timeout(10.minutes, JobTimeoutError) do
|
118
232
|
block.call
|
119
233
|
end
|
120
234
|
end
|
121
235
|
end
|
122
236
|
```
|
123
|
-
|
124
|
-
### Configuring
|
237
|
+
|
238
|
+
### Configuring job execution threads
|
125
239
|
|
126
240
|
GoodJob executes enqueued jobs using threads. There is a lot than can be said about [multithreaded behavior in Ruby on Rails](https://guides.rubyonrails.org/threading_and_code_execution.html), but briefly:
|
127
241
|
|
@@ -132,6 +246,51 @@ GoodJob executes enqueued jobs using threads. There is a lot than can be said ab
|
|
132
246
|
3. `$ RAILS_MAX_THREADS=4 bundle exec good_job`
|
133
247
|
4. Implicitly via Rails's database connection pool size (`ActiveRecord::Base.connection_pool.size`)
|
134
248
|
|
249
|
+
### Executing jobs async / in-process
|
250
|
+
|
251
|
+
GoodJob is able to run "async" in the same process as the webserver (e.g. `bin/rail s`). GoodJob's async execution mode offers benefits of economy by not requiring a separate job worker process, but with the tradeoff of increased complexity. Async mode can be configured in two ways:
|
252
|
+
|
253
|
+
- Directly configure the ActiveJob adapter:
|
254
|
+
|
255
|
+
```ruby
|
256
|
+
# config/environments/production.rb
|
257
|
+
config.active_job.queue_adapter = GoodJob::Adapter.new(execution_mode: :async, max_threads: 4, poll_interval: 30)
|
258
|
+
```
|
259
|
+
- Or, when using `...queue_adapter = :good_job`, via environment variables:
|
260
|
+
|
261
|
+
```bash
|
262
|
+
$ GOOD_JOB_EXECUTION_MODE=async GOOD_JOB_MAX_THREADS=4 GOOD_JOB_POLL_INTERVAL=30 bin/rails server
|
263
|
+
```
|
264
|
+
|
265
|
+
Depending on your application configuration, you may need to take additional steps:
|
266
|
+
|
267
|
+
- Ensure that you have enough database connections for both web and job execution threads:
|
268
|
+
|
269
|
+
```yaml
|
270
|
+
# config/database.yml
|
271
|
+
pool: <%= ENV.fetch("RAILS_MAX_THREADS", 5).to_i + ENV.fetch("GOOD_JOB_MAX_THREADS", 4).to_i %>
|
272
|
+
```
|
273
|
+
|
274
|
+
- When running Puma with workers (`WEB_CONCURRENCY > 0`) or another process-forking webserver, GoodJob's threadpool schedulers should be stopped before forking, restarted after fork, and cleanly shut down on exit. Stopping GoodJob's scheduler pre-fork is recommended to ensure that GoodJob does not continue executing jobs in the parent/controller process. For example, with Puma:
|
275
|
+
|
276
|
+
```ruby
|
277
|
+
# config/puma.rb
|
278
|
+
|
279
|
+
before_fork do
|
280
|
+
GoodJob::Scheduler.instances.each { |s| s.shutdown }
|
281
|
+
end
|
282
|
+
|
283
|
+
on_worker_boot do
|
284
|
+
GoodJob::Scheduler.instances.each { |s| s.restart }
|
285
|
+
end
|
286
|
+
|
287
|
+
on_worker_shutdown do
|
288
|
+
GoodJob::Scheduler.instances.each { |s| s.shutdown }
|
289
|
+
end
|
290
|
+
```
|
291
|
+
|
292
|
+
GoodJob is compatible with Puma's `preload_app!` method.
|
293
|
+
|
135
294
|
### Migrating to GoodJob from a different ActiveJob backend
|
136
295
|
|
137
296
|
If your application is already using an ActiveJob backend, you will need to install GoodJob to enqueue and perform newly created jobs _and_ finish performing pre-existing jobs on the previous backend.
|
@@ -147,7 +306,8 @@ If your application is already using an ActiveJob backend, you will need to inst
|
|
147
306
|
```
|
148
307
|
|
149
308
|
1. Continue running executors for both backends. For example, on Heroku it's possible to run [two processes](https://help.heroku.com/CTFS2TJK/how-do-i-run-multiple-processes-on-a-dyno) within the same dyno:
|
150
|
-
|
309
|
+
|
310
|
+
```procfile
|
151
311
|
# Procfile
|
152
312
|
# ...
|
153
313
|
worker: bundle exec que ./config/environment.rb & bundle exec good_job & wait -n
|
@@ -173,15 +333,24 @@ It is also necessary to delete these preserved jobs from the database after a ce
|
|
173
333
|
- For example, in a Rake task:
|
174
334
|
|
175
335
|
```ruby
|
176
|
-
|
336
|
+
GoodJob::Job.finished(1.day.ago).delete_all
|
177
337
|
```
|
338
|
+
|
178
339
|
- For example, using the `good_job` command-line utility:
|
179
340
|
|
180
341
|
```bash
|
181
342
|
$ bundle exec good_job cleanup_preserved_jobs --before-seconds-ago=86400
|
182
343
|
```
|
183
344
|
|
184
|
-
##
|
345
|
+
## Contributing
|
346
|
+
|
347
|
+
Contributions are welcomed and appreciated 🙏
|
348
|
+
|
349
|
+
- Review the [Prioritized Project Backlog](https://github.com/bensheldon/good_job/projects/1).
|
350
|
+
- Open a new Issue or contribute to an [existing Issue](https://github.com/bensheldon/good_job/issues). Questions or suggestions are fantastic.
|
351
|
+
- Participate according to our [Code of Conduct](https://github.com/bensheldon/good_job/projects/1).
|
352
|
+
|
353
|
+
### Gem development
|
185
354
|
|
186
355
|
To run tests:
|
187
356
|
|
@@ -190,7 +359,7 @@ To run tests:
|
|
190
359
|
$ git clone git@github.com:bensheldon/good_job.git
|
191
360
|
|
192
361
|
# Set up the local environment
|
193
|
-
$ bin/
|
362
|
+
$ bin/setup
|
194
363
|
|
195
364
|
# Run the tests
|
196
365
|
$ bin/rspec
|
@@ -204,7 +373,6 @@ $ bundle exec appraisal
|
|
204
373
|
|
205
374
|
# Run tests
|
206
375
|
$ bundle exec appraisal bin/rspec
|
207
|
-
|
208
376
|
```
|
209
377
|
|
210
378
|
For developing locally within another Ruby on Rails project:
|
@@ -219,24 +387,23 @@ $ bundle install
|
|
219
387
|
# => Using good_job 0.1.0 from https://github.com/bensheldon/good_job.git (at /Users/You/Projects/good_job@dc57fb0)
|
220
388
|
```
|
221
389
|
|
222
|
-
|
390
|
+
### Releasing
|
223
391
|
|
224
|
-
Package maintainers can release this gem
|
392
|
+
Package maintainers can release this gem by running:
|
225
393
|
|
226
394
|
```bash
|
227
395
|
# Sign into rubygems
|
228
396
|
$ gem signin
|
229
397
|
|
398
|
+
# Add a .env file with the following:
|
399
|
+
# CHANGELOG_GITHUB_TOKEN= # Github Personal Access Token
|
400
|
+
|
230
401
|
# Update version number, changelog, and create git commit:
|
231
|
-
$ bundle exec rake
|
402
|
+
$ bundle exec rake release[minor] # major,minor,patch
|
232
403
|
|
233
404
|
# ..and follow subsequent directions.
|
234
405
|
```
|
235
406
|
|
236
|
-
## Contributing
|
237
|
-
|
238
|
-
Contribution directions go here.
|
239
|
-
|
240
407
|
## License
|
241
408
|
|
242
409
|
The gem is available as open source under the terms of the [MIT License](https://opensource.org/licenses/MIT).
|