good_job 2.6.1 → 2.7.2
Sign up to get free protection for your applications and to get access to all the features.
- checksums.yaml +4 -4
- data/CHANGELOG.md +56 -0
- data/README.md +102 -32
- data/engine/app/charts/good_job/scheduled_by_queue_chart.rb +4 -1
- data/engine/app/controllers/good_job/base_controller.rb +19 -0
- data/engine/app/filters/good_job/executions_filter.rb +1 -1
- data/engine/app/filters/good_job/jobs_filter.rb +1 -1
- data/lib/good_job/adapter.rb +34 -11
- data/lib/good_job/cli.rb +9 -0
- data/lib/good_job/configuration.rb +23 -18
- data/lib/good_job/cron_manager.rb +1 -1
- data/lib/good_job/filterable.rb +2 -2
- data/lib/good_job/job_performer.rb +2 -5
- data/lib/good_job/lockable.rb +16 -8
- data/lib/good_job/notifier.rb +1 -1
- data/lib/good_job/poller.rb +1 -1
- data/lib/good_job/probe_server.rb +51 -0
- data/lib/good_job/railtie.rb +14 -4
- data/lib/good_job/scheduler.rb +2 -2
- data/lib/good_job/version.rb +1 -1
- data/lib/good_job.rb +10 -1
- metadata +17 -2
checksums.yaml
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
---
|
2
2
|
SHA256:
|
3
|
-
metadata.gz:
|
4
|
-
data.tar.gz:
|
3
|
+
metadata.gz: e4843101f6fce50527e8d0157ff7ced91fabb0ae41c3c92006abd49e9005ea78
|
4
|
+
data.tar.gz: f8f243fb7f9e3ea2ec17fc795ad5bbd79b017f68665d98b911dfe27177e13389
|
5
5
|
SHA512:
|
6
|
-
metadata.gz:
|
7
|
-
data.tar.gz:
|
6
|
+
metadata.gz: f2dbdd1c1811bfc09153a8eed32b95f337919d22434b004622a0ab78eb6803e68f5a66761e828bf2054421d5d5f9b798be54e7f60cc486602a997c804ab9fb2d
|
7
|
+
data.tar.gz: 953672a066c27a231fb76c3c9839b4cb9685deb0ed4f25a3e153f4ede0833afa0a263f4d37532715a62ed7a96825384b8f7d6fef7352416d0c64068864798fee
|
data/CHANGELOG.md
CHANGED
@@ -1,5 +1,61 @@
|
|
1
1
|
# Changelog
|
2
2
|
|
3
|
+
## [v2.7.2](https://github.com/bensheldon/good_job/tree/v2.7.2) (2021-11-29)
|
4
|
+
|
5
|
+
[Full Changelog](https://github.com/bensheldon/good_job/compare/v2.7.1...v2.7.2)
|
6
|
+
|
7
|
+
**Implemented enhancements:**
|
8
|
+
|
9
|
+
- Allow GoodJob global configuration accessors to also be set via Rails config hash [\#460](https://github.com/bensheldon/good_job/pull/460) ([bensheldon](https://github.com/bensheldon))
|
10
|
+
|
11
|
+
**Merged pull requests:**
|
12
|
+
|
13
|
+
- Use `ActiveRecord::Relation::QueryAttribute` when setting up bindings for `exec_query` [\#461](https://github.com/bensheldon/good_job/pull/461) ([bensheldon](https://github.com/bensheldon))
|
14
|
+
- Configure RSpec `config.example_status_persistence_file_path` [\#459](https://github.com/bensheldon/good_job/pull/459) ([bensheldon](https://github.com/bensheldon))
|
15
|
+
- Defer async initialization until Rails fully initialized [\#454](https://github.com/bensheldon/good_job/pull/454) ([bensheldon](https://github.com/bensheldon))
|
16
|
+
|
17
|
+
## [v2.7.1](https://github.com/bensheldon/good_job/tree/v2.7.1) (2021-11-26)
|
18
|
+
|
19
|
+
[Full Changelog](https://github.com/bensheldon/good_job/compare/v2.7.0...v2.7.1)
|
20
|
+
|
21
|
+
**Fixed bugs:**
|
22
|
+
|
23
|
+
- Unclear error when database can't be reached [\#457](https://github.com/bensheldon/good_job/issues/457)
|
24
|
+
- Remove Concurrent::Delay wrapping of database-loading methods [\#458](https://github.com/bensheldon/good_job/pull/458) ([bensheldon](https://github.com/bensheldon))
|
25
|
+
- Do not delete csp policies when checking csp policies [\#456](https://github.com/bensheldon/good_job/pull/456) ([JonathanFrias](https://github.com/JonathanFrias))
|
26
|
+
|
27
|
+
**Closed issues:**
|
28
|
+
|
29
|
+
- How to suppress job scheduler logs? [\#455](https://github.com/bensheldon/good_job/issues/455)
|
30
|
+
- Configuration in environments/\*.rb overrides application.rb [\#453](https://github.com/bensheldon/good_job/issues/453)
|
31
|
+
- Testing jobs synchronously [\#435](https://github.com/bensheldon/good_job/issues/435)
|
32
|
+
- HTTP health check endpoint [\#403](https://github.com/bensheldon/good_job/issues/403)
|
33
|
+
|
34
|
+
## [v2.7.0](https://github.com/bensheldon/good_job/tree/v2.7.0) (2021-11-10)
|
35
|
+
|
36
|
+
[Full Changelog](https://github.com/bensheldon/good_job/compare/v2.6.2...v2.7.0)
|
37
|
+
|
38
|
+
**Implemented enhancements:**
|
39
|
+
|
40
|
+
- Add http probe for CLI healthcheck/readiness/liveliness [\#452](https://github.com/bensheldon/good_job/pull/452) ([bensheldon](https://github.com/bensheldon))
|
41
|
+
- Add explicit Content Security Policy \(CSP\) for Dashboard [\#449](https://github.com/bensheldon/good_job/pull/449) ([bensheldon](https://github.com/bensheldon))
|
42
|
+
|
43
|
+
**Closed issues:**
|
44
|
+
|
45
|
+
- Add a default Content-Security-Policy for the Dashboard [\#420](https://github.com/bensheldon/good_job/issues/420)
|
46
|
+
|
47
|
+
## [v2.6.2](https://github.com/bensheldon/good_job/tree/v2.6.2) (2021-11-05)
|
48
|
+
|
49
|
+
[Full Changelog](https://github.com/bensheldon/good_job/compare/v2.6.1...v2.6.2)
|
50
|
+
|
51
|
+
**Fixed bugs:**
|
52
|
+
|
53
|
+
- Rename Filterable\#search to Filterable\#search\_text to avoid name collision [\#451](https://github.com/bensheldon/good_job/pull/451) ([bensheldon](https://github.com/bensheldon))
|
54
|
+
|
55
|
+
**Closed issues:**
|
56
|
+
|
57
|
+
- v2.6.1 is incompatible with gem thinking-sphinx [\#450](https://github.com/bensheldon/good_job/issues/450)
|
58
|
+
|
3
59
|
## [v2.6.1](https://github.com/bensheldon/good_job/tree/v2.6.1) (2021-11-05)
|
4
60
|
|
5
61
|
[Full Changelog](https://github.com/bensheldon/good_job/compare/v2.6.0...v2.6.1)
|
data/README.md
CHANGED
@@ -55,6 +55,7 @@ For more of the story of GoodJob, read the [introductory blog post](https://isla
|
|
55
55
|
- [Migrate to GoodJob from a different ActiveJob backend](#migrate-to-goodjob-from-a-different-activejob-backend)
|
56
56
|
- [Monitor and preserve worked jobs](#monitor-and-preserve-worked-jobs)
|
57
57
|
- [PgBouncer compatibility](#pgbouncer-compatibility)
|
58
|
+
- [CLI HTTP health check probes](#cli-http-health-check-probes)
|
58
59
|
- [Contribute](#contribute)
|
59
60
|
- [Gem development](#gem-development)
|
60
61
|
- [Release](#release)
|
@@ -96,7 +97,7 @@ For more of the story of GoodJob, read the [introductory blog post](https://isla
|
|
96
97
|
1. Configure the ActiveJob adapter:
|
97
98
|
|
98
99
|
```ruby
|
99
|
-
# config/application.rb
|
100
|
+
# config/application.rb or config/environments/{RAILS_ENV}.rb
|
100
101
|
config.active_job.queue_adapter = :good_job
|
101
102
|
```
|
102
103
|
|
@@ -170,6 +171,7 @@ Options:
|
|
170
171
|
[--enable-cron] # Whether to run cron process (default: false)
|
171
172
|
[--daemonize] # Run as a background daemon (default: false)
|
172
173
|
[--pidfile=PIDFILE] # Path to write daemonized Process ID (env var: GOOD_JOB_PIDFILE, default: tmp/pids/good_job.pid)
|
174
|
+
[--probe-port=PORT] # Port for http health check (env var: GOOD_JOB_PROBE_PORT, default: nil)
|
173
175
|
|
174
176
|
Executes queued jobs.
|
175
177
|
|
@@ -210,43 +212,48 @@ to delete old records and preserve space in your database.
|
|
210
212
|
|
211
213
|
### Configuration options
|
212
214
|
|
213
|
-
|
215
|
+
ActiveJob configuration depends on where the code is placed:
|
214
216
|
|
215
|
-
|
217
|
+
- `config.active_job.queue_adapter = :good_job` within `config/application.rb` or `config/environments/*.rb`.
|
218
|
+
- `ActiveJob::Base.queue_adapter = :good_job` within an initializer (e.g. `config/initializers/active_job.rb`).
|
216
219
|
|
217
|
-
|
220
|
+
GoodJob configuration can be placed within Rails `config` directory for all environments (`config/application.rb`), within a particular environment (e.g. `config/environments/development.rb`), or within an initializer (e.g. `config/initializers/good_job.rb`).
|
218
221
|
|
219
222
|
Configuration examples:
|
220
223
|
|
221
224
|
```ruby
|
222
|
-
|
223
|
-
|
224
|
-
config.
|
225
|
-
|
226
|
-
|
227
|
-
config.good_job.execution_mode = :async
|
228
|
-
config.good_job.max_threads = 5
|
229
|
-
config.good_job.poll_interval = 30 # seconds
|
230
|
-
config.good_job.shutdown_timeout = 25 # seconds
|
231
|
-
config.good_job.enable_cron = true
|
232
|
-
config.good_job.cron = { example: { cron: '0 * * * *', class: 'ExampleJob' } }
|
233
|
-
config.good_job.queues = '*'
|
234
|
-
|
235
|
-
# ...or all at once.
|
236
|
-
config.good_job = {
|
237
|
-
|
238
|
-
|
239
|
-
|
240
|
-
|
241
|
-
|
242
|
-
|
243
|
-
|
244
|
-
|
245
|
-
|
225
|
+
Rails.application.configure do
|
226
|
+
# Configure options individually...
|
227
|
+
config.good_job.preserve_job_records = true
|
228
|
+
config.good_job.retry_on_unhandled_error = false
|
229
|
+
config.good_job.on_thread_error = -> (exception) { Raven.capture_exception(exception) }
|
230
|
+
config.good_job.execution_mode = :async
|
231
|
+
config.good_job.max_threads = 5
|
232
|
+
config.good_job.poll_interval = 30 # seconds
|
233
|
+
config.good_job.shutdown_timeout = 25 # seconds
|
234
|
+
config.good_job.enable_cron = true
|
235
|
+
config.good_job.cron = { example: { cron: '0 * * * *', class: 'ExampleJob' } }
|
236
|
+
config.good_job.queues = '*'
|
237
|
+
|
238
|
+
# ...or all at once.
|
239
|
+
config.good_job = {
|
240
|
+
preserve_job_records: true,
|
241
|
+
retry_on_unhandled_error: false,
|
242
|
+
on_thread_error: -> (exception) { Raven.capture_exception(exception) },
|
243
|
+
execution_mode: :async,
|
244
|
+
max_threads: 5,
|
245
|
+
poll_interval: 30,
|
246
|
+
shutdown_timeout: 25,
|
247
|
+
enable_cron: true,
|
248
|
+
cron: {
|
249
|
+
example: {
|
250
|
+
cron: '0 * * * *',
|
251
|
+
class: 'ExampleJob'
|
252
|
+
},
|
246
253
|
},
|
247
|
-
|
248
|
-
|
249
|
-
|
254
|
+
queues: '*',
|
255
|
+
}
|
256
|
+
end
|
250
257
|
```
|
251
258
|
|
252
259
|
Available configuration options are:
|
@@ -263,6 +270,14 @@ Available configuration options are:
|
|
263
270
|
- `shutdown_timeout` (float) number of seconds to wait for jobs to finish when shutting down before stopping the thread. Defaults to forever: `-1`. You can also set this with the environment variable `GOOD_JOB_SHUTDOWN_TIMEOUT`.
|
264
271
|
- `enable_cron` (boolean) whether to run cron process. Defaults to `false`. You can also set this with the environment variable `GOOD_JOB_ENABLE_CRON`.
|
265
272
|
- `cron` (hash) cron configuration. Defaults to `{}`. You can also set this as a JSON string with the environment variable `GOOD_JOB_CRON`
|
273
|
+
- `logger` ([Rails Logger](https://api.rubyonrails.org/classes/ActiveSupport/Logger.html)) lets you set a custom logger for GoodJob. It should be an instance of a Rails `Logger` (Default: `Rails.logger`).
|
274
|
+
- `preserve_job_records` (boolean) keeps job records in your database even after jobs are completed. (Default: `false`)
|
275
|
+
- `retry_on_unhandled_error` (boolean) causes jobs to be re-queued and retried if they raise an instance of `StandardError`. Instances of `Exception`, like SIGINT, will *always* be retried, regardless of this attribute’s value. (Default: `true`)
|
276
|
+
- `on_thread_error` (proc, lambda, or callable) will be called when an Exception. It can be useful for logging errors to bug tracking services, like Sentry or Airbrake. Example:
|
277
|
+
|
278
|
+
```ruby
|
279
|
+
config.good_job.on_thread_error = -> (exception) { Raven.capture_exception(exception) }
|
280
|
+
```
|
266
281
|
|
267
282
|
By default, GoodJob configures the following execution modes per environment:
|
268
283
|
|
@@ -283,7 +298,7 @@ config.good_job.execution_mode = :external
|
|
283
298
|
|
284
299
|
### Global options
|
285
300
|
|
286
|
-
Good Job’s general behavior can also be configured via
|
301
|
+
Good Job’s general behavior can also be configured via attributes directly on the `GoodJob` module:
|
287
302
|
|
288
303
|
- **`GoodJob.active_record_parent_class`** (string) The ActiveRecord parent class inherited by GoodJob's ActiveRecord model `GoodJob::Job` (defaults to `"ActiveRecord::Base"`). Configure this when using [multiple databases with ActiveRecord](https://guides.rubyonrails.org/active_record_multiple_databases.html) or when other custom configuration is necessary for the ActiveRecord model to connect to the Postgres database. _The value must be a String to avoid premature initialization of ActiveRecord._
|
289
304
|
- **`GoodJob.logger`** ([Rails Logger](https://api.rubyonrails.org/classes/ActiveSupport/Logger.html)) lets you set a custom logger for GoodJob. It should be an instance of a Rails `Logger`.
|
@@ -837,6 +852,61 @@ A workaround to this limitation is to make a direct database connection availabl
|
|
837
852
|
GoodJob.active_record_parent_class = "ApplicationDirectRecord"
|
838
853
|
```
|
839
854
|
|
855
|
+
### CLI HTTP health check probes
|
856
|
+
|
857
|
+
GoodJob's CLI offers an http health check probe to better manage process lifecycle in containerized environments like Kubernetes:
|
858
|
+
|
859
|
+
```bash
|
860
|
+
# Run the CLI with a health check on port 7001
|
861
|
+
good_job start --probe-port=7001
|
862
|
+
|
863
|
+
# or via an environment variable
|
864
|
+
GOOD_JOB_PROBE_PORT=7001 good_job start
|
865
|
+
|
866
|
+
# Probe the status
|
867
|
+
curl localhost:7001/status
|
868
|
+
curl localhost:7001/status/started
|
869
|
+
curl localhost:7001/status/connected
|
870
|
+
```
|
871
|
+
|
872
|
+
Multiple health checks are available at different paths:
|
873
|
+
|
874
|
+
- `/` or `/status`: the CLI process is running
|
875
|
+
- `/status/started`: the multithreaded job executor is running
|
876
|
+
- `/status/connected`: the database connection is established
|
877
|
+
|
878
|
+
This can be configured, for example with Kubernetes:
|
879
|
+
|
880
|
+
```yaml
|
881
|
+
spec:
|
882
|
+
containers:
|
883
|
+
- name: good_job
|
884
|
+
image: my_app:latest
|
885
|
+
env:
|
886
|
+
- name: RAILS_ENV
|
887
|
+
value: production
|
888
|
+
- name: GOOD_JOB_PROBE_PORT
|
889
|
+
value: 7001
|
890
|
+
command:
|
891
|
+
- good_job
|
892
|
+
- start
|
893
|
+
ports:
|
894
|
+
- name: probe-port
|
895
|
+
containerPort: 7001
|
896
|
+
startupProbe:
|
897
|
+
httpGet:
|
898
|
+
path: "/status/started"
|
899
|
+
port: probe-port
|
900
|
+
failureThreshold: 30
|
901
|
+
periodSeconds: 10
|
902
|
+
livenessProbe:
|
903
|
+
httpGet:
|
904
|
+
path: "/status/connected"
|
905
|
+
port: probe-port
|
906
|
+
failureThreshold: 1
|
907
|
+
periodSeconds: 10
|
908
|
+
```
|
909
|
+
|
840
910
|
## Contribute
|
841
911
|
|
842
912
|
Contributions are welcomed and appreciated 🙏
|
@@ -30,7 +30,10 @@ module GoodJob
|
|
30
30
|
ORDER BY timestamp ASC
|
31
31
|
SQL
|
32
32
|
|
33
|
-
binds = [
|
33
|
+
binds = [
|
34
|
+
ActiveRecord::Relation::QueryAttribute.new('start_time', start_time, ActiveRecord::Type::DateTime.new),
|
35
|
+
ActiveRecord::Relation::QueryAttribute.new('end_time', end_time, ActiveRecord::Type::DateTime.new),
|
36
|
+
]
|
34
37
|
executions_data = GoodJob::Execution.connection.exec_query(GoodJob::Execution.pg_or_jdbc_query(count_query), "GoodJob Dashboard Chart", binds)
|
35
38
|
|
36
39
|
queue_names = executions_data.reject { |d| d['count'].nil? }.map { |d| d['queue_name'] || BaseFilter::EMPTY }.uniq
|
@@ -5,6 +5,25 @@ module GoodJob
|
|
5
5
|
|
6
6
|
around_action :switch_locale
|
7
7
|
|
8
|
+
content_security_policy do |policy|
|
9
|
+
policy.default_src(:none) if policy.default_src(*policy.default_src).blank?
|
10
|
+
policy.connect_src(:self) if policy.connect_src(*policy.connect_src).blank?
|
11
|
+
policy.base_uri(:none) if policy.base_uri(*policy.base_uri).blank?
|
12
|
+
policy.font_src(:self) if policy.font_src(*policy.font_src).blank?
|
13
|
+
policy.img_src(:self, :data) if policy.img_src(*policy.img_src).blank?
|
14
|
+
policy.object_src(:none) if policy.object_src(*policy.object_src).blank?
|
15
|
+
policy.script_src(:self) if policy.script_src(*policy.script_src).blank?
|
16
|
+
policy.style_src(:self) if policy.style_src(*policy.style_src).blank?
|
17
|
+
policy.form_action(:self) if policy.form_action(*policy.form_action).blank?
|
18
|
+
policy.frame_ancestors(:none) if policy.frame_ancestors(*policy.frame_ancestors).blank?
|
19
|
+
end
|
20
|
+
|
21
|
+
before_action do
|
22
|
+
next if request.content_security_policy_nonce_generator
|
23
|
+
|
24
|
+
request.content_security_policy_nonce_generator = ->(_request) { SecureRandom.base64(16) }
|
25
|
+
end
|
26
|
+
|
8
27
|
private
|
9
28
|
|
10
29
|
def switch_locale(&action)
|
@@ -14,7 +14,7 @@ module GoodJob
|
|
14
14
|
query = base_query
|
15
15
|
query = query.job_class(params[:job_class]) if params[:job_class].present?
|
16
16
|
query = query.where(queue_name: params[:queue_name]) if params[:queue_name].present?
|
17
|
-
query = query.
|
17
|
+
query = query.search_text(params[:query]) if params[:query].present?
|
18
18
|
|
19
19
|
if params[:state]
|
20
20
|
case params[:state]
|
@@ -18,7 +18,7 @@ module GoodJob
|
|
18
18
|
|
19
19
|
query = query.job_class(params[:job_class]) if params[:job_class].present?
|
20
20
|
query = query.where(queue_name: params[:queue_name]) if params[:queue_name].present?
|
21
|
-
query = query.
|
21
|
+
query = query.search_text(params[:query]) if params[:query].present?
|
22
22
|
|
23
23
|
if params[:state]
|
24
24
|
case params[:state]
|
data/lib/good_job/adapter.rb
CHANGED
@@ -4,6 +4,12 @@ module GoodJob
|
|
4
4
|
# ActiveJob Adapter.
|
5
5
|
#
|
6
6
|
class Adapter
|
7
|
+
# @!attribute [r] instances
|
8
|
+
# @!scope class
|
9
|
+
# List of all instantiated Adapters in the current process.
|
10
|
+
# @return [Array<GoodJob::Adapter>, nil]
|
11
|
+
cattr_reader :instances, default: [], instance_reader: false
|
12
|
+
|
7
13
|
# @param execution_mode [Symbol, nil] specifies how and where jobs should be executed. You can also set this with the environment variable +GOOD_JOB_EXECUTION_MODE+.
|
8
14
|
#
|
9
15
|
# - +:inline+ executes jobs immediately in whatever process queued them (usually the web server process). This should only be used in test and development environments.
|
@@ -20,7 +26,8 @@ module GoodJob
|
|
20
26
|
# @param max_threads [Integer, nil] sets the number of threads per scheduler to use when +execution_mode+ is set to +:async+. The +queues+ parameter can specify a number of threads for each group of queues which will override this value. You can also set this with the environment variable +GOOD_JOB_MAX_THREADS+. Defaults to +5+.
|
21
27
|
# @param queues [String, nil] determines which queues to execute jobs from when +execution_mode+ is set to +:async+. See {file:README.md#optimize-queues-threads-and-processes} for more details on the format of this string. You can also set this with the environment variable +GOOD_JOB_QUEUES+. Defaults to +"*"+.
|
22
28
|
# @param poll_interval [Integer, nil] sets the number of seconds between polls for jobs when +execution_mode+ is set to +:async+. You can also set this with the environment variable +GOOD_JOB_POLL_INTERVAL+. Defaults to +1+.
|
23
|
-
|
29
|
+
# @param start_async_on_initialize [Boolean] whether to start the async scheduler when the adapter is initialized.
|
30
|
+
def initialize(execution_mode: nil, queues: nil, max_threads: nil, poll_interval: nil, start_async_on_initialize: Rails.application.initialized?)
|
24
31
|
@configuration = GoodJob::Configuration.new(
|
25
32
|
{
|
26
33
|
execution_mode: execution_mode,
|
@@ -30,16 +37,9 @@ module GoodJob
|
|
30
37
|
}
|
31
38
|
)
|
32
39
|
@configuration.validate!
|
40
|
+
self.class.instances << self
|
33
41
|
|
34
|
-
if
|
35
|
-
@notifier = GoodJob::Notifier.new
|
36
|
-
@poller = GoodJob::Poller.new(poll_interval: @configuration.poll_interval)
|
37
|
-
@scheduler = GoodJob::Scheduler.from_configuration(@configuration, warm_cache_on_initialize: Rails.application.initialized?)
|
38
|
-
@notifier.recipients << [@scheduler, :create_thread]
|
39
|
-
@poller.recipients << [@scheduler, :create_thread]
|
40
|
-
|
41
|
-
@cron_manager = GoodJob::CronManager.new(@configuration.cron_entries, start_on_initialize: Rails.application.initialized?) if @configuration.enable_cron?
|
42
|
-
end
|
42
|
+
start_async if start_async_on_initialize
|
43
43
|
end
|
44
44
|
|
45
45
|
# Enqueues the ActiveJob job to be performed.
|
@@ -74,7 +74,7 @@ module GoodJob
|
|
74
74
|
job_state = { queue_name: execution.queue_name }
|
75
75
|
job_state[:scheduled_at] = execution.scheduled_at if execution.scheduled_at
|
76
76
|
|
77
|
-
executed_locally = execute_async? && @scheduler
|
77
|
+
executed_locally = execute_async? && @scheduler&.create_thread(job_state)
|
78
78
|
Notifier.notify(job_state) unless executed_locally
|
79
79
|
end
|
80
80
|
|
@@ -97,6 +97,7 @@ module GoodJob
|
|
97
97
|
|
98
98
|
executables = [@notifier, @poller, @scheduler].compact
|
99
99
|
GoodJob._shutdown_all(executables, timeout: timeout)
|
100
|
+
@_async_started = false
|
100
101
|
end
|
101
102
|
|
102
103
|
# Whether in +:async+ execution mode.
|
@@ -119,6 +120,28 @@ module GoodJob
|
|
119
120
|
@configuration.execution_mode == :inline
|
120
121
|
end
|
121
122
|
|
123
|
+
# Start async executors
|
124
|
+
# @return void
|
125
|
+
def start_async
|
126
|
+
return unless execute_async?
|
127
|
+
|
128
|
+
@notifier = GoodJob::Notifier.new
|
129
|
+
@poller = GoodJob::Poller.new(poll_interval: @configuration.poll_interval)
|
130
|
+
@scheduler = GoodJob::Scheduler.from_configuration(@configuration, warm_cache_on_initialize: true)
|
131
|
+
@notifier.recipients << [@scheduler, :create_thread]
|
132
|
+
@poller.recipients << [@scheduler, :create_thread]
|
133
|
+
|
134
|
+
@cron_manager = GoodJob::CronManager.new(@configuration.cron_entries, start_on_initialize: true) if @configuration.enable_cron?
|
135
|
+
|
136
|
+
@_async_started = true
|
137
|
+
end
|
138
|
+
|
139
|
+
# Whether the async executors are running
|
140
|
+
# @return [Boolean]
|
141
|
+
def async_started?
|
142
|
+
@_async_started
|
143
|
+
end
|
144
|
+
|
122
145
|
private
|
123
146
|
|
124
147
|
# Whether running in a web server process.
|
data/lib/good_job/cli.rb
CHANGED
@@ -79,6 +79,9 @@ module GoodJob
|
|
79
79
|
method_option :pidfile,
|
80
80
|
type: :string,
|
81
81
|
desc: "Path to write daemonized Process ID (env var: GOOD_JOB_PIDFILE, default: tmp/pids/good_job.pid)"
|
82
|
+
method_option :probe_port,
|
83
|
+
type: :numeric,
|
84
|
+
desc: "Port for http health check (env var: GOOD_JOB_PROBE_PORT, default: nil)"
|
82
85
|
|
83
86
|
def start
|
84
87
|
set_up_application!
|
@@ -94,6 +97,11 @@ module GoodJob
|
|
94
97
|
|
95
98
|
cron_manager = GoodJob::CronManager.new(configuration.cron_entries, start_on_initialize: true) if configuration.enable_cron?
|
96
99
|
|
100
|
+
if configuration.probe_port
|
101
|
+
probe_server = GoodJob::ProbeServer.new(port: configuration.probe_port)
|
102
|
+
probe_server.start
|
103
|
+
end
|
104
|
+
|
97
105
|
@stop_good_job_executable = false
|
98
106
|
%w[INT TERM].each do |signal|
|
99
107
|
trap(signal) { @stop_good_job_executable = true }
|
@@ -106,6 +114,7 @@ module GoodJob
|
|
106
114
|
|
107
115
|
executors = [notifier, poller, cron_manager, scheduler].compact
|
108
116
|
GoodJob._shutdown_all(executors, timeout: configuration.shutdown_timeout)
|
117
|
+
probe_server&.stop
|
109
118
|
end
|
110
119
|
|
111
120
|
default_task :start
|
@@ -50,24 +50,22 @@ module GoodJob
|
|
50
50
|
# for more details on possible values.
|
51
51
|
# @return [Symbol]
|
52
52
|
def execution_mode
|
53
|
-
|
54
|
-
|
55
|
-
|
56
|
-
|
57
|
-
|
58
|
-
|
59
|
-
|
60
|
-
|
61
|
-
|
62
|
-
|
63
|
-
|
64
|
-
|
65
|
-
|
66
|
-
|
67
|
-
|
68
|
-
|
69
|
-
:external
|
70
|
-
end
|
53
|
+
mode = if GoodJob::CLI.within_exe?
|
54
|
+
:external
|
55
|
+
else
|
56
|
+
options[:execution_mode] ||
|
57
|
+
rails_config[:execution_mode] ||
|
58
|
+
env['GOOD_JOB_EXECUTION_MODE']
|
59
|
+
end
|
60
|
+
|
61
|
+
if mode
|
62
|
+
mode.to_sym
|
63
|
+
elsif Rails.env.development?
|
64
|
+
:async
|
65
|
+
elsif Rails.env.test?
|
66
|
+
:inline
|
67
|
+
else
|
68
|
+
:external
|
71
69
|
end
|
72
70
|
end
|
73
71
|
|
@@ -195,6 +193,13 @@ module GoodJob
|
|
195
193
|
Rails.application.root.join('tmp', 'pids', 'good_job.pid')
|
196
194
|
end
|
197
195
|
|
196
|
+
# Port of the probe server
|
197
|
+
# @return [nil,Integer]
|
198
|
+
def probe_port
|
199
|
+
options[:probe_port] ||
|
200
|
+
env['GOOD_JOB_PROBE_PORT']
|
201
|
+
end
|
202
|
+
|
198
203
|
private
|
199
204
|
|
200
205
|
def rails_config
|
@@ -21,7 +21,7 @@ module GoodJob # :nodoc:
|
|
21
21
|
def self.task_observer(time, output, thread_error) # rubocop:disable Lint/UnusedMethodArgument
|
22
22
|
return if thread_error.is_a? Concurrent::CancelledOperationError
|
23
23
|
|
24
|
-
GoodJob.
|
24
|
+
GoodJob._on_thread_error(thread_error) if thread_error
|
25
25
|
end
|
26
26
|
|
27
27
|
# Execution configuration to be scheduled
|
data/lib/good_job/filterable.rb
CHANGED
@@ -24,12 +24,12 @@ module GoodJob
|
|
24
24
|
end)
|
25
25
|
|
26
26
|
# Search records by text query.
|
27
|
-
# @!method
|
27
|
+
# @!method search_text(query)
|
28
28
|
# @!scope class
|
29
29
|
# @param query [String]
|
30
30
|
# Search Query
|
31
31
|
# @return [ActiveRecord::Relation]
|
32
|
-
scope :
|
32
|
+
scope :search_text, (lambda do |query|
|
33
33
|
query = query.to_s.strip
|
34
34
|
next if query.blank?
|
35
35
|
|
@@ -13,9 +13,6 @@ module GoodJob
|
|
13
13
|
# @param queue_string [String] Queues to execute jobs from
|
14
14
|
def initialize(queue_string)
|
15
15
|
@queue_string = queue_string
|
16
|
-
|
17
|
-
@job_query = Concurrent::Delay.new { GoodJob::Execution.queue_string(queue_string) }
|
18
|
-
@parsed_queues = Concurrent::Delay.new { GoodJob::Execution.queue_parser(queue_string) }
|
19
16
|
end
|
20
17
|
|
21
18
|
# A meaningful name to identify the performer in logs and for debugging.
|
@@ -65,11 +62,11 @@ module GoodJob
|
|
65
62
|
attr_reader :queue_string
|
66
63
|
|
67
64
|
def job_query
|
68
|
-
@
|
65
|
+
@_job_query ||= GoodJob::Execution.queue_string(queue_string)
|
69
66
|
end
|
70
67
|
|
71
68
|
def parsed_queues
|
72
|
-
@
|
69
|
+
@_parsed_queues ||= GoodJob::Execution.queue_parser(queue_string)
|
73
70
|
end
|
74
71
|
end
|
75
72
|
end
|
data/lib/good_job/lockable.rb
CHANGED
@@ -24,7 +24,7 @@ module GoodJob
|
|
24
24
|
|
25
25
|
included do
|
26
26
|
# Default column to be used when creating Advisory Locks
|
27
|
-
class_attribute :advisory_lockable_column, instance_accessor: false, default:
|
27
|
+
class_attribute :advisory_lockable_column, instance_accessor: false, default: nil
|
28
28
|
|
29
29
|
# Default Postgres function to be used for Advisory Locks
|
30
30
|
class_attribute :advisory_lockable_function, default: "pg_try_advisory_lock"
|
@@ -161,10 +161,8 @@ module GoodJob
|
|
161
161
|
end
|
162
162
|
end
|
163
163
|
|
164
|
-
# Allow advisory_lockable_column to be a `Concurrent::Delay`
|
165
164
|
def _advisory_lockable_column
|
166
|
-
|
167
|
-
column.respond_to?(:value) ? column.value : column
|
165
|
+
advisory_lockable_column || primary_key
|
168
166
|
end
|
169
167
|
|
170
168
|
def supports_cte_materialization_specifiers?
|
@@ -217,7 +215,9 @@ module GoodJob
|
|
217
215
|
SQL
|
218
216
|
end
|
219
217
|
|
220
|
-
binds = [
|
218
|
+
binds = [
|
219
|
+
ActiveRecord::Relation::QueryAttribute.new('key', key, ActiveRecord::Type::String.new),
|
220
|
+
]
|
221
221
|
self.class.connection.exec_query(pg_or_jdbc_query(query), 'GoodJob::Lockable Advisory Lock', binds).first['locked']
|
222
222
|
end
|
223
223
|
|
@@ -231,7 +231,9 @@ module GoodJob
|
|
231
231
|
query = <<~SQL.squish
|
232
232
|
SELECT #{function}(('x'||substr(md5($1::text), 1, 16))::bit(64)::bigint) AS unlocked
|
233
233
|
SQL
|
234
|
-
binds = [
|
234
|
+
binds = [
|
235
|
+
ActiveRecord::Relation::QueryAttribute.new('key', key, ActiveRecord::Type::String.new),
|
236
|
+
]
|
235
237
|
self.class.connection.exec_query(pg_or_jdbc_query(query), 'GoodJob::Lockable Advisory Unlock', binds).first['unlocked']
|
236
238
|
end
|
237
239
|
|
@@ -281,7 +283,10 @@ module GoodJob
|
|
281
283
|
AND pg_locks.classid = ('x' || substr(md5($1::text), 1, 16))::bit(32)::int
|
282
284
|
AND pg_locks.objid = (('x' || substr(md5($2::text), 1, 16))::bit(64) << 32)::bit(32)::int
|
283
285
|
SQL
|
284
|
-
binds = [
|
286
|
+
binds = [
|
287
|
+
ActiveRecord::Relation::QueryAttribute.new('key', key, ActiveRecord::Type::String.new),
|
288
|
+
ActiveRecord::Relation::QueryAttribute.new('key', key, ActiveRecord::Type::String.new),
|
289
|
+
]
|
285
290
|
self.class.connection.exec_query(pg_or_jdbc_query(query), 'GoodJob::Lockable Advisory Locked?', binds).any?
|
286
291
|
end
|
287
292
|
|
@@ -305,7 +310,10 @@ module GoodJob
|
|
305
310
|
AND pg_locks.objid = (('x' || substr(md5($2::text), 1, 16))::bit(64) << 32)::bit(32)::int
|
306
311
|
AND pg_locks.pid = pg_backend_pid()
|
307
312
|
SQL
|
308
|
-
binds = [
|
313
|
+
binds = [
|
314
|
+
ActiveRecord::Relation::QueryAttribute.new('key', key, ActiveRecord::Type::String.new),
|
315
|
+
ActiveRecord::Relation::QueryAttribute.new('key', key, ActiveRecord::Type::String.new),
|
316
|
+
]
|
309
317
|
self.class.connection.exec_query(pg_or_jdbc_query(query), 'GoodJob::Lockable Owns Advisory Lock?', binds).any?
|
310
318
|
end
|
311
319
|
|
data/lib/good_job/notifier.rb
CHANGED
@@ -120,7 +120,7 @@ module GoodJob # :nodoc:
|
|
120
120
|
return if thread_error.is_a? AdapterCannotListenError
|
121
121
|
|
122
122
|
if thread_error
|
123
|
-
GoodJob.
|
123
|
+
GoodJob._on_thread_error(thread_error)
|
124
124
|
ActiveSupport::Notifications.instrument("notifier_notify_error.good_job", { error: thread_error })
|
125
125
|
|
126
126
|
connection_error = CONNECTION_ERRORS.any? do |error_string|
|
data/lib/good_job/poller.rb
CHANGED
@@ -91,7 +91,7 @@ module GoodJob # :nodoc:
|
|
91
91
|
# @param thread_error [Exception, nil]
|
92
92
|
# @return [void]
|
93
93
|
def timer_observer(time, executed_task, thread_error)
|
94
|
-
GoodJob.
|
94
|
+
GoodJob._on_thread_error(thread_error) if thread_error
|
95
95
|
ActiveSupport::Notifications.instrument("finished_timer_task", { result: executed_task, error: thread_error, time: time })
|
96
96
|
end
|
97
97
|
|
@@ -0,0 +1,51 @@
|
|
1
|
+
# frozen_string_literal: true
|
2
|
+
|
3
|
+
module GoodJob
|
4
|
+
class ProbeServer
|
5
|
+
RACK_SERVER = 'webrick'
|
6
|
+
|
7
|
+
def self.task_observer(time, output, thread_error) # rubocop:disable Lint/UnusedMethodArgument
|
8
|
+
return if thread_error.is_a? Concurrent::CancelledOperationError
|
9
|
+
|
10
|
+
GoodJob._on_thread_error(thread_error) if thread_error
|
11
|
+
end
|
12
|
+
|
13
|
+
def initialize(port:)
|
14
|
+
@port = port
|
15
|
+
end
|
16
|
+
|
17
|
+
def start
|
18
|
+
@handler = Rack::Handler.get(RACK_SERVER)
|
19
|
+
@future = Concurrent::Future.new(args: [@handler, @port, GoodJob.logger]) do |thr_handler, thr_port, thr_logger|
|
20
|
+
thr_handler.run(self, Port: thr_port, Logger: thr_logger, AccessLog: [])
|
21
|
+
end
|
22
|
+
@future.add_observer(self.class, :task_observer)
|
23
|
+
@future.execute
|
24
|
+
end
|
25
|
+
|
26
|
+
def running?
|
27
|
+
@handler&.instance_variable_get(:@server)&.status == :Running
|
28
|
+
end
|
29
|
+
|
30
|
+
def stop
|
31
|
+
@handler&.shutdown
|
32
|
+
@future&.value # wait for Future to exit
|
33
|
+
end
|
34
|
+
|
35
|
+
def call(env)
|
36
|
+
case Rack::Request.new(env).path
|
37
|
+
when '/', '/status'
|
38
|
+
[200, {}, ["OK"]]
|
39
|
+
when '/status/started'
|
40
|
+
started = GoodJob::Scheduler.instances.any? && GoodJob::Scheduler.instances.all?(&:running?)
|
41
|
+
started ? [200, {}, ["Started"]] : [503, {}, ["Not started"]]
|
42
|
+
when '/status/connected'
|
43
|
+
connected = GoodJob::Scheduler.instances.any? && GoodJob::Scheduler.instances.all?(&:running?) &&
|
44
|
+
GoodJob::Notifier.instances.any? && GoodJob::Notifier.instances.all?(&:listening?)
|
45
|
+
connected ? [200, {}, ["Connected"]] : [503, {}, ["Not connected"]]
|
46
|
+
else
|
47
|
+
[404, {}, ["Not found"]]
|
48
|
+
end
|
49
|
+
end
|
50
|
+
end
|
51
|
+
end
|
data/lib/good_job/railtie.rb
CHANGED
@@ -7,7 +7,7 @@ module GoodJob
|
|
7
7
|
|
8
8
|
initializer "good_job.logger" do |_app|
|
9
9
|
ActiveSupport.on_load(:good_job) do
|
10
|
-
self.logger = ::Rails.logger
|
10
|
+
self.logger = ::Rails.logger if GoodJob.logger == GoodJob::DEFAULT_LOGGER
|
11
11
|
end
|
12
12
|
GoodJob::LogSubscriber.attach_to :good_job
|
13
13
|
end
|
@@ -22,9 +22,19 @@ module GoodJob
|
|
22
22
|
end
|
23
23
|
end
|
24
24
|
|
25
|
-
|
26
|
-
|
27
|
-
|
25
|
+
initializer 'good_job.rails_config' do
|
26
|
+
config.after_initialize do
|
27
|
+
GoodJob.logger = Rails.application.config.good_job.logger unless Rails.application.config.good_job.logger.nil?
|
28
|
+
GoodJob.on_thread_error = Rails.application.config.good_job.on_thread_error unless Rails.application.config.good_job.on_thread_error.nil?
|
29
|
+
GoodJob.preserve_job_records = Rails.application.config.good_job.preserve_job_records unless Rails.application.config.good_job.preserve_job_records.nil?
|
30
|
+
GoodJob.retry_on_unhandled_error = Rails.application.config.good_job.retry_on_unhandled_error unless Rails.application.config.good_job.retry_on_unhandled_error.nil?
|
31
|
+
end
|
32
|
+
end
|
33
|
+
|
34
|
+
initializer "good_job.start_async" do
|
35
|
+
config.after_initialize do
|
36
|
+
GoodJob::Adapter.instances.each(&:start_async)
|
37
|
+
end
|
28
38
|
end
|
29
39
|
end
|
30
40
|
end
|
data/lib/good_job/scheduler.rb
CHANGED
@@ -169,7 +169,7 @@ module GoodJob # :nodoc:
|
|
169
169
|
# @return [void]
|
170
170
|
def task_observer(time, output, thread_error)
|
171
171
|
error = thread_error || (output.is_a?(GoodJob::ExecutionResult) ? output.unhandled_error : nil)
|
172
|
-
GoodJob.
|
172
|
+
GoodJob._on_thread_error(error) if error
|
173
173
|
|
174
174
|
instrument("finished_job_task", { result: output, error: thread_error, time: time })
|
175
175
|
create_task if output
|
@@ -206,7 +206,7 @@ module GoodJob # :nodoc:
|
|
206
206
|
end
|
207
207
|
|
208
208
|
observer = lambda do |_time, _output, thread_error|
|
209
|
-
GoodJob.
|
209
|
+
GoodJob._on_thread_error(thread_error) if thread_error
|
210
210
|
create_task # If cache-warming exhausts the threads, ensure there isn't an executable task remaining
|
211
211
|
end
|
212
212
|
future.add_observer(observer, :call)
|
data/lib/good_job/version.rb
CHANGED
data/lib/good_job.rb
CHANGED
@@ -18,6 +18,8 @@ require "good_job/railtie"
|
|
18
18
|
#
|
19
19
|
# +GoodJob+ is the top-level namespace and exposes configuration attributes.
|
20
20
|
module GoodJob
|
21
|
+
DEFAULT_LOGGER = ActiveSupport::TaggedLogging.new(ActiveSupport::Logger.new($stdout))
|
22
|
+
|
21
23
|
# @!attribute [rw] active_record_parent_class
|
22
24
|
# @!scope class
|
23
25
|
# The ActiveRecord parent class inherited by +GoodJob::Execution+ (default: +ActiveRecord::Base+).
|
@@ -34,7 +36,7 @@ module GoodJob
|
|
34
36
|
# @return [Logger, nil]
|
35
37
|
# @example Output GoodJob logs to a file:
|
36
38
|
# GoodJob.logger = ActiveSupport::TaggedLogging.new(ActiveSupport::Logger.new("log/my_logs.log"))
|
37
|
-
mattr_accessor :logger, default:
|
39
|
+
mattr_accessor :logger, default: DEFAULT_LOGGER
|
38
40
|
|
39
41
|
# @!attribute [rw] preserve_job_records
|
40
42
|
# @!scope class
|
@@ -66,6 +68,13 @@ module GoodJob
|
|
66
68
|
# @return [Proc, nil]
|
67
69
|
mattr_accessor :on_thread_error, default: nil
|
68
70
|
|
71
|
+
# Called with exception when a GoodJob thread raises an exception
|
72
|
+
# @param exception [Exception] Exception that was raised
|
73
|
+
# @return [void]
|
74
|
+
def self._on_thread_error(exception)
|
75
|
+
on_thread_error.call(exception) if on_thread_error.respond_to?(:call)
|
76
|
+
end
|
77
|
+
|
69
78
|
# Stop executing jobs.
|
70
79
|
# GoodJob does its work in pools of background threads.
|
71
80
|
# When forking processes you should shut down these background threads before forking, and restart them after forking.
|
metadata
CHANGED
@@ -1,14 +1,14 @@
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
2
2
|
name: good_job
|
3
3
|
version: !ruby/object:Gem::Version
|
4
|
-
version: 2.
|
4
|
+
version: 2.7.2
|
5
5
|
platform: ruby
|
6
6
|
authors:
|
7
7
|
- Ben Sheldon
|
8
8
|
autorequire:
|
9
9
|
bindir: exe
|
10
10
|
cert_chain: []
|
11
|
-
date: 2021-11-
|
11
|
+
date: 2021-11-29 00:00:00.000000000 Z
|
12
12
|
dependencies:
|
13
13
|
- !ruby/object:Gem::Dependency
|
14
14
|
name: activejob
|
@@ -94,6 +94,20 @@ dependencies:
|
|
94
94
|
- - ">="
|
95
95
|
- !ruby/object:Gem::Version
|
96
96
|
version: 0.14.1
|
97
|
+
- !ruby/object:Gem::Dependency
|
98
|
+
name: webrick
|
99
|
+
requirement: !ruby/object:Gem::Requirement
|
100
|
+
requirements:
|
101
|
+
- - ">="
|
102
|
+
- !ruby/object:Gem::Version
|
103
|
+
version: '1.3'
|
104
|
+
type: :runtime
|
105
|
+
prerelease: false
|
106
|
+
version_requirements: !ruby/object:Gem::Requirement
|
107
|
+
requirements:
|
108
|
+
- - ">="
|
109
|
+
- !ruby/object:Gem::Version
|
110
|
+
version: '1.3'
|
97
111
|
- !ruby/object:Gem::Dependency
|
98
112
|
name: zeitwerk
|
99
113
|
requirement: !ruby/object:Gem::Requirement
|
@@ -410,6 +424,7 @@ files:
|
|
410
424
|
- lib/good_job/multi_scheduler.rb
|
411
425
|
- lib/good_job/notifier.rb
|
412
426
|
- lib/good_job/poller.rb
|
427
|
+
- lib/good_job/probe_server.rb
|
413
428
|
- lib/good_job/railtie.rb
|
414
429
|
- lib/good_job/scheduler.rb
|
415
430
|
- lib/good_job/version.rb
|