good_job 1.10.1 → 1.11.3
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/CHANGELOG.md +67 -2
- data/README.md +30 -0
- data/engine/app/controllers/good_job/active_jobs_controller.rb +1 -0
- data/engine/app/controllers/good_job/assets_controller.rb +1 -0
- data/engine/app/controllers/good_job/base_controller.rb +1 -0
- data/engine/app/controllers/good_job/dashboards_controller.rb +1 -0
- data/engine/app/controllers/good_job/jobs_controller.rb +1 -0
- data/engine/app/helpers/good_job/application_helper.rb +1 -0
- data/engine/app/views/layouts/good_job/base.html.erb +5 -5
- data/engine/config/routes.rb +11 -5
- data/engine/lib/good_job/engine.rb +1 -0
- data/exe/good_job +1 -0
- data/lib/active_job/queue_adapters/good_job_adapter.rb +1 -0
- data/lib/generators/good_job/install_generator.rb +1 -0
- data/lib/generators/good_job/templates/update/migrations/01_create_good_jobs.rb +1 -0
- data/lib/generators/good_job/templates/update/migrations/02_add_active_job_id_concurrency_key_cron_key_to_good_jobs.rb +1 -0
- data/lib/generators/good_job/templates/update/migrations/03_add_active_job_id_index_and_concurrency_key_index_to_good_jobs.rb +1 -0
- data/lib/generators/good_job/update_generator.rb +1 -0
- data/lib/good_job.rb +1 -0
- data/lib/good_job/active_job_extensions.rb +5 -0
- data/lib/good_job/active_job_extensions/concurrency.rb +69 -0
- data/lib/good_job/adapter.rb +1 -0
- data/lib/good_job/cli.rb +1 -0
- data/lib/good_job/configuration.rb +1 -0
- data/lib/good_job/current_execution.rb +13 -5
- data/lib/good_job/daemon.rb +1 -0
- data/lib/good_job/execution_result.rb +1 -0
- data/lib/good_job/job.rb +24 -2
- data/lib/good_job/job_performer.rb +1 -0
- data/lib/good_job/lockable.rb +32 -18
- data/lib/good_job/log_subscriber.rb +1 -0
- data/lib/good_job/multi_scheduler.rb +1 -0
- data/lib/good_job/notifier.rb +13 -4
- data/lib/good_job/poller.rb +1 -0
- data/lib/good_job/railtie.rb +1 -0
- data/lib/good_job/scheduler.rb +1 -0
- data/lib/good_job/version.rb +2 -1
- metadata +4 -2
checksums.yaml
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
---
|
2
2
|
SHA256:
|
3
|
-
metadata.gz:
|
4
|
-
data.tar.gz:
|
3
|
+
metadata.gz: 5e0f0ca8d7f8b024f5e84b515b8603d362b6c755d54680221e517369b6877c07
|
4
|
+
data.tar.gz: e69518c9d71d7b08e3707c9df8c6429ca4d9823fd1adea32adc6c0bf41ad33f4
|
5
5
|
SHA512:
|
6
|
-
metadata.gz:
|
7
|
-
data.tar.gz:
|
6
|
+
metadata.gz: 50d2542bfad082dfba516c08bd994dad3fcd6e8311a8380dc012988c3e2745378271adc7d41cb91897d087a6b2b100cfb69362d3a11cd55c92c2610ceabeb80c
|
7
|
+
data.tar.gz: 69ffc79dc9367fc8522bdcf669cede6791d4e7903847edbc99887404b29ec43361de54a960841ee6cdeaaf236cc77d8c9c60b2c61793b5256bf2455667e0c387
|
data/CHANGELOG.md
CHANGED
@@ -1,12 +1,77 @@
|
|
1
1
|
# Changelog
|
2
2
|
|
3
|
+
## [v1.11.3](https://github.com/bensheldon/good_job/tree/v1.11.3) (2021-07-25)
|
4
|
+
|
5
|
+
[Full Changelog](https://github.com/bensheldon/good_job/compare/v1.11.2...v1.11.3)
|
6
|
+
|
7
|
+
**Closed issues:**
|
8
|
+
|
9
|
+
- ERROR: relation "good\_jobs" does not exist at character 454 [\#308](https://github.com/bensheldon/good_job/issues/308)
|
10
|
+
- Add Frozen String Literal to all files [\#298](https://github.com/bensheldon/good_job/issues/298)
|
11
|
+
- Support for good\_job without Rails? [\#295](https://github.com/bensheldon/good_job/issues/295)
|
12
|
+
|
13
|
+
**Merged pull requests:**
|
14
|
+
|
15
|
+
- Have prettier Dashboard asset urls e.g. `bootstrap.css` instead of `bootstrap_css.css` [\#306](https://github.com/bensheldon/good_job/pull/306) ([bensheldon](https://github.com/bensheldon))
|
16
|
+
- Create dashboard demo app on Heroku [\#305](https://github.com/bensheldon/good_job/pull/305) ([bensheldon](https://github.com/bensheldon))
|
17
|
+
- Add Frozen String Literal to all files [\#302](https://github.com/bensheldon/good_job/pull/302) ([tedhexaflow](https://github.com/tedhexaflow))
|
18
|
+
|
19
|
+
## [v1.11.2](https://github.com/bensheldon/good_job/tree/v1.11.2) (2021-07-20)
|
20
|
+
|
21
|
+
[Full Changelog](https://github.com/bensheldon/good_job/compare/v1.11.1...v1.11.2)
|
22
|
+
|
23
|
+
**Fixed bugs:**
|
24
|
+
|
25
|
+
- Notifier waits to retry listening when database is unavailable [\#301](https://github.com/bensheldon/good_job/pull/301) ([bensheldon](https://github.com/bensheldon))
|
26
|
+
|
27
|
+
**Closed issues:**
|
28
|
+
|
29
|
+
- Handle database connection drops [\#296](https://github.com/bensheldon/good_job/issues/296)
|
30
|
+
- Using the `async` worker results in `ActiveModel::UnknownAttributeError unknown attribute 'create_with_advisory_lock' for GoodJob::Job`. [\#290](https://github.com/bensheldon/good_job/issues/290)
|
31
|
+
|
32
|
+
**Merged pull requests:**
|
33
|
+
|
34
|
+
- Rename development and test databases to be `good_job` [\#300](https://github.com/bensheldon/good_job/pull/300) ([bensheldon](https://github.com/bensheldon))
|
35
|
+
- Move generators spec into top-level spec directory; update dependencies [\#299](https://github.com/bensheldon/good_job/pull/299) ([bensheldon](https://github.com/bensheldon))
|
36
|
+
|
37
|
+
## [v1.11.1](https://github.com/bensheldon/good_job/tree/v1.11.1) (2021-07-07)
|
38
|
+
|
39
|
+
[Full Changelog](https://github.com/bensheldon/good_job/compare/v1.11.0...v1.11.1)
|
40
|
+
|
41
|
+
**Fixed bugs:**
|
42
|
+
|
43
|
+
- Defer accessing ActiveRecord `primary_key` in Lockable [\#293](https://github.com/bensheldon/good_job/pull/293) ([bensheldon](https://github.com/bensheldon))
|
44
|
+
|
45
|
+
**Closed issues:**
|
46
|
+
|
47
|
+
- Database connection required while loading the code on 1.10.x [\#291](https://github.com/bensheldon/good_job/issues/291)
|
48
|
+
|
49
|
+
## [v1.11.0](https://github.com/bensheldon/good_job/tree/v1.11.0) (2021-07-07)
|
50
|
+
|
51
|
+
[Full Changelog](https://github.com/bensheldon/good_job/compare/v1.10.1...v1.11.0)
|
52
|
+
|
53
|
+
**Implemented enhancements:**
|
54
|
+
|
55
|
+
- Add concurrency extension for ActiveJob [\#281](https://github.com/bensheldon/good_job/pull/281) ([bensheldon](https://github.com/bensheldon))
|
56
|
+
|
57
|
+
**Closed issues:**
|
58
|
+
|
59
|
+
- Investigate GoodJob concurrency [\#289](https://github.com/bensheldon/good_job/issues/289)
|
60
|
+
- Problem with migrating database on 1.10.0 [\#287](https://github.com/bensheldon/good_job/issues/287)
|
61
|
+
- Support migration --database option for install task? [\#267](https://github.com/bensheldon/good_job/issues/267)
|
62
|
+
- Add GoodJob to Ruby Toolbox [\#243](https://github.com/bensheldon/good_job/issues/243)
|
63
|
+
- Custom advisory locks to prevent certain jobs from being worked on concurrently? [\#206](https://github.com/bensheldon/good_job/issues/206)
|
64
|
+
|
3
65
|
## [v1.10.1](https://github.com/bensheldon/good_job/tree/v1.10.1) (2021-06-30)
|
4
66
|
|
5
67
|
[Full Changelog](https://github.com/bensheldon/good_job/compare/v1.10.0...v1.10.1)
|
6
68
|
|
7
|
-
**
|
69
|
+
**Fixed bugs:**
|
8
70
|
|
9
71
|
- Remove `FOR UPDATE SKIP LOCKED` from job locking sql statement [\#288](https://github.com/bensheldon/good_job/pull/288) ([bensheldon](https://github.com/bensheldon))
|
72
|
+
|
73
|
+
**Merged pull requests:**
|
74
|
+
|
10
75
|
- Update GH Test Matrix with latest JRuby 9.2.19.0 [\#283](https://github.com/bensheldon/good_job/pull/283) ([tedhexaflow](https://github.com/tedhexaflow))
|
11
76
|
|
12
77
|
## [v1.10.0](https://github.com/bensheldon/good_job/tree/v1.10.0) (2021-06-29)
|
@@ -15,11 +80,11 @@
|
|
15
80
|
|
16
81
|
**Implemented enhancements:**
|
17
82
|
|
83
|
+
- Use `pg_advisory_unlock_all` after each thread's job execution; fix Lockable return values; improve test stability [\#285](https://github.com/bensheldon/good_job/pull/285) ([bensheldon](https://github.com/bensheldon))
|
18
84
|
- Add `rails g good_job:update` command to add idempotent migration files, including `active_job_id`, `concurrency_key`, `cron_key` columns [\#266](https://github.com/bensheldon/good_job/pull/266) ([bensheldon](https://github.com/bensheldon))
|
19
85
|
|
20
86
|
**Fixed bugs:**
|
21
87
|
|
22
|
-
- Use `pg_advisory_unlock_all` after each thread's job execution; fix Lockable return values; improve test stability [\#285](https://github.com/bensheldon/good_job/pull/285) ([bensheldon](https://github.com/bensheldon))
|
23
88
|
- Dashboard AssetsController does not raise if verify\_authenticity\_token is not in the callback chain [\#284](https://github.com/bensheldon/good_job/pull/284) ([bensheldon](https://github.com/bensheldon))
|
24
89
|
|
25
90
|
**Closed issues:**
|
data/README.md
CHANGED
@@ -38,6 +38,7 @@ For more of the story of GoodJob, read the [introductory blog post](https://isla
|
|
38
38
|
- [Configuration options](#configuration-options)
|
39
39
|
- [Global options](#global-options)
|
40
40
|
- [Dashboard](#dashboard)
|
41
|
+
- [ActiveJob Concurrency](#activejob-concurrency)
|
41
42
|
- [Updating](#updating)
|
42
43
|
- [Go deeper](#go-deeper)
|
43
44
|
- [Exceptions, retries, and reliability](#exceptions-retries-and-reliability)
|
@@ -319,6 +320,35 @@ GoodJob includes a Dashboard as a mountable `Rails::Engine`.
|
|
319
320
|
end
|
320
321
|
```
|
321
322
|
|
323
|
+
### ActiveJob Concurrency
|
324
|
+
|
325
|
+
GoodJob can extend ActiveJob to provide limits on concurrently running jobs, either at time of _enqueue_ or at _perform_.
|
326
|
+
|
327
|
+
**Note:** Limiting concurrency at _enqueue_ requires Rails 6.0+ because Rails 5.2 does not support `throw :abort` in ActiveJob callbacks.
|
328
|
+
|
329
|
+
```ruby
|
330
|
+
class MyJob < ApplicationJob
|
331
|
+
include GoodJob::ActiveJobExtensions::Concurrency
|
332
|
+
|
333
|
+
good_job_control_concurrency_with(
|
334
|
+
# Maximum number of jobs with the concurrency key to be concurrently enqueued
|
335
|
+
enqueue_limit: 2,
|
336
|
+
|
337
|
+
# Maximum number of jobs with the concurrency key to be concurrently performed
|
338
|
+
perform_limit: 1,
|
339
|
+
|
340
|
+
# A unique key to be globally locked against.
|
341
|
+
# Can be String or Lambda/Proc that is invoked in the context of the job.
|
342
|
+
# Note: Arguments passed to #perform_later must be accessed through `arguments` method.
|
343
|
+
key: -> { "Unique-#{arguments.first}" } # MyJob.perform_later("Alice") => "Unique-Alice"
|
344
|
+
)
|
345
|
+
|
346
|
+
def perform(first_name)
|
347
|
+
# do work
|
348
|
+
end
|
349
|
+
end
|
350
|
+
```
|
351
|
+
|
322
352
|
### Updating
|
323
353
|
|
324
354
|
GoodJob follows semantic versioning, though updates may be encouraged through deprecation warnings in minor versions.
|
@@ -5,12 +5,12 @@
|
|
5
5
|
<%= csrf_meta_tags %>
|
6
6
|
<%= csp_meta_tag %>
|
7
7
|
|
8
|
-
<%= stylesheet_link_tag
|
9
|
-
<%= stylesheet_link_tag
|
10
|
-
<%= stylesheet_link_tag
|
8
|
+
<%= stylesheet_link_tag bootstrap_path(format: :css, v: GoodJob::VERSION) %>
|
9
|
+
<%= stylesheet_link_tag chartist_path(format: :css, v: GoodJob::VERSION) %>
|
10
|
+
<%= stylesheet_link_tag style_path(format: :css, v: GoodJob::VERSION) %>
|
11
11
|
|
12
|
-
<%= javascript_include_tag
|
13
|
-
<%= javascript_include_tag
|
12
|
+
<%= javascript_include_tag bootstrap_path(format: :js, v: GoodJob::VERSION) %>
|
13
|
+
<%= javascript_include_tag chartist_path(format: :js, v: GoodJob::VERSION) %>
|
14
14
|
</head>
|
15
15
|
<body>
|
16
16
|
<nav class="navbar navbar-expand-lg navbar-light bg-light">
|
data/engine/config/routes.rb
CHANGED
@@ -1,13 +1,19 @@
|
|
1
|
+
# frozen_string_literal: true
|
1
2
|
GoodJob::Engine.routes.draw do
|
2
3
|
root to: 'dashboards#index'
|
3
4
|
resources :active_jobs, only: %i[show]
|
4
5
|
resources :jobs, only: %i[destroy]
|
5
6
|
|
6
7
|
scope controller: :assets do
|
7
|
-
|
8
|
-
|
9
|
-
|
10
|
-
|
11
|
-
|
8
|
+
constraints(format: :css) do
|
9
|
+
get :bootstrap, action: :bootstrap_css
|
10
|
+
get :chartist, action: :chartist_css
|
11
|
+
get :style, action: :style_css
|
12
|
+
end
|
13
|
+
|
14
|
+
constraints(format: :js) do
|
15
|
+
get :bootstrap, action: :bootstrap_js
|
16
|
+
get :chartist, action: :chartist_js
|
17
|
+
end
|
12
18
|
end
|
13
19
|
end
|
data/exe/good_job
CHANGED
data/lib/good_job.rb
CHANGED
@@ -0,0 +1,69 @@
|
|
1
|
+
# frozen_string_literal: true
|
2
|
+
module GoodJob
|
3
|
+
module ActiveJobExtensions
|
4
|
+
module Concurrency
|
5
|
+
extend ActiveSupport::Concern
|
6
|
+
|
7
|
+
ConcurrencyExceededError = Class.new(StandardError)
|
8
|
+
|
9
|
+
included do
|
10
|
+
class_attribute :good_job_concurrency_config, instance_accessor: false, default: {}
|
11
|
+
|
12
|
+
before_enqueue do |job|
|
13
|
+
# Always allow jobs to be retried because the current job's execution will complete momentarily
|
14
|
+
next if CurrentExecution.active_job_id == job.job_id
|
15
|
+
|
16
|
+
limit = job.class.good_job_concurrency_config.fetch(:enqueue_limit, Float::INFINITY)
|
17
|
+
next if limit.blank? || (0...Float::INFINITY).exclude?(limit)
|
18
|
+
|
19
|
+
key = job.good_job_concurrency_key
|
20
|
+
next if key.blank?
|
21
|
+
|
22
|
+
GoodJob::Job.new.with_advisory_lock(key: key, function: "pg_advisory_lock") do
|
23
|
+
# TODO: Why is `unscoped` necessary? Nested scope is bleeding into subsequent query?
|
24
|
+
enqueue_concurrency = GoodJob::Job.unscoped.where(concurrency_key: key).unfinished.count
|
25
|
+
# The job has not yet been enqueued, so check if adding it will go over the limit
|
26
|
+
throw :abort if enqueue_concurrency + 1 > limit
|
27
|
+
end
|
28
|
+
end
|
29
|
+
|
30
|
+
retry_on(
|
31
|
+
GoodJob::ActiveJobExtensions::Concurrency::ConcurrencyExceededError,
|
32
|
+
attempts: Float::INFINITY,
|
33
|
+
wait: :exponentially_longer
|
34
|
+
)
|
35
|
+
|
36
|
+
before_perform do |job|
|
37
|
+
limit = job.class.good_job_concurrency_config.fetch(:perform_limit, Float::INFINITY)
|
38
|
+
next if limit.blank? || (0...Float::INFINITY).exclude?(limit)
|
39
|
+
|
40
|
+
key = job.good_job_concurrency_key
|
41
|
+
next if key.blank?
|
42
|
+
|
43
|
+
GoodJob::Job.new.with_advisory_lock(key: key, function: "pg_advisory_lock") do
|
44
|
+
perform_concurrency = GoodJob::Job.unscoped.where(concurrency_key: key).advisory_locked.count
|
45
|
+
# The current job has already been locked and will appear in the previous query
|
46
|
+
raise GoodJob::ActiveJobExtensions::Concurrency::ConcurrencyExceededError if perform_concurrency > limit
|
47
|
+
end
|
48
|
+
end
|
49
|
+
end
|
50
|
+
|
51
|
+
class_methods do
|
52
|
+
def good_job_control_concurrency_with(config)
|
53
|
+
self.good_job_concurrency_config = config
|
54
|
+
end
|
55
|
+
end
|
56
|
+
|
57
|
+
def good_job_concurrency_key
|
58
|
+
key = self.class.good_job_concurrency_config[:key]
|
59
|
+
return if key.blank?
|
60
|
+
|
61
|
+
if key.respond_to? :call
|
62
|
+
instance_exec(&key)
|
63
|
+
else
|
64
|
+
key
|
65
|
+
end
|
66
|
+
end
|
67
|
+
end
|
68
|
+
end
|
69
|
+
end
|
data/lib/good_job/adapter.rb
CHANGED
data/lib/good_job/cli.rb
CHANGED
@@ -1,14 +1,15 @@
|
|
1
|
+
# frozen_string_literal: true
|
1
2
|
require 'active_support/core_ext/module/attribute_accessors_per_thread'
|
2
3
|
|
3
4
|
module GoodJob
|
4
5
|
# Thread-local attributes for passing values from Instrumentation.
|
5
6
|
# (Cannot use ActiveSupport::CurrentAttributes because ActiveJob resets it)
|
6
7
|
module CurrentExecution
|
7
|
-
# @!attribute [rw]
|
8
|
+
# @!attribute [rw] active_job_id
|
8
9
|
# @!scope class
|
9
|
-
#
|
10
|
-
# @return [
|
11
|
-
thread_mattr_accessor :
|
10
|
+
# ActiveJob ID
|
11
|
+
# @return [String, nil]
|
12
|
+
thread_mattr_accessor :active_job_id
|
12
13
|
|
13
14
|
# @!attribute [rw] error_on_discard
|
14
15
|
# @!scope class
|
@@ -16,11 +17,18 @@ module GoodJob
|
|
16
17
|
# @return [Exception, nil]
|
17
18
|
thread_mattr_accessor :error_on_discard
|
18
19
|
|
20
|
+
# @!attribute [rw] error_on_retry
|
21
|
+
# @!scope class
|
22
|
+
# Error captured by retry_on
|
23
|
+
# @return [Exception, nil]
|
24
|
+
thread_mattr_accessor :error_on_retry
|
25
|
+
|
19
26
|
# Resets attributes
|
20
27
|
# @return [void]
|
21
28
|
def self.reset
|
22
|
-
self.
|
29
|
+
self.active_job_id = nil
|
23
30
|
self.error_on_discard = nil
|
31
|
+
self.error_on_retry = nil
|
24
32
|
end
|
25
33
|
|
26
34
|
# @return [Integer] Current process ID
|
data/lib/good_job/daemon.rb
CHANGED
data/lib/good_job/job.rb
CHANGED
@@ -1,3 +1,4 @@
|
|
1
|
+
# frozen_string_literal: true
|
1
2
|
module GoodJob
|
2
3
|
# ActiveRecord model that represents an +ActiveJob+ job.
|
3
4
|
# Parent class can be configured with +GoodJob.active_record_parent_class+.
|
@@ -10,11 +11,12 @@ module GoodJob
|
|
10
11
|
PreviouslyPerformedError = Class.new(StandardError)
|
11
12
|
|
12
13
|
# ActiveJob jobs without a +queue_name+ attribute are placed on this queue.
|
13
|
-
DEFAULT_QUEUE_NAME = 'default'
|
14
|
+
DEFAULT_QUEUE_NAME = 'default'
|
14
15
|
# ActiveJob jobs without a +priority+ attribute are given this priority.
|
15
16
|
DEFAULT_PRIORITY = 0
|
16
17
|
|
17
|
-
self.table_name = 'good_jobs'
|
18
|
+
self.table_name = 'good_jobs'
|
19
|
+
self.advisory_lockable_column = 'id'
|
18
20
|
|
19
21
|
attr_readonly :serialized_params
|
20
22
|
|
@@ -219,6 +221,21 @@ module GoodJob
|
|
219
221
|
DEPRECATION
|
220
222
|
end
|
221
223
|
|
224
|
+
if column_names.include?('concurrency_key')
|
225
|
+
good_job_args[:concurrency_key] = active_job.good_job_concurrency_key if active_job.respond_to?(:good_job_concurrency_key)
|
226
|
+
else
|
227
|
+
ActiveSupport::Deprecation.warn(<<~DEPRECATION)
|
228
|
+
GoodJob has pending database migrations. To create the migration files, run:
|
229
|
+
|
230
|
+
rails generate good_job:update
|
231
|
+
|
232
|
+
To apply the migration files, run:
|
233
|
+
|
234
|
+
rails db:migrate
|
235
|
+
|
236
|
+
DEPRECATION
|
237
|
+
end
|
238
|
+
|
222
239
|
good_job = GoodJob::Job.new(**good_job_args)
|
223
240
|
|
224
241
|
instrument_payload[:good_job] = good_job
|
@@ -264,6 +281,10 @@ module GoodJob
|
|
264
281
|
self.class.unscoped.unfinished.owns_advisory_locked.exists?(id: id)
|
265
282
|
end
|
266
283
|
|
284
|
+
def active_job_id
|
285
|
+
super || serialized_params['job_id']
|
286
|
+
end
|
287
|
+
|
267
288
|
private
|
268
289
|
|
269
290
|
# @return [ExecutionResult]
|
@@ -273,6 +294,7 @@ module GoodJob
|
|
273
294
|
)
|
274
295
|
|
275
296
|
GoodJob::CurrentExecution.reset
|
297
|
+
GoodJob::CurrentExecution.active_job_id = active_job_id
|
276
298
|
ActiveSupport::Notifications.instrument("perform_job.good_job", { good_job: self, process_id: GoodJob::CurrentExecution.process_id, thread_name: GoodJob::CurrentExecution.thread_name }) do
|
277
299
|
value = ActiveJob::Base.execute(params)
|
278
300
|
|
data/lib/good_job/lockable.rb
CHANGED
@@ -1,3 +1,4 @@
|
|
1
|
+
# frozen_string_literal: true
|
1
2
|
module GoodJob
|
2
3
|
#
|
3
4
|
# Adds Postgres advisory locking capabilities to an ActiveRecord record.
|
@@ -23,20 +24,20 @@ module GoodJob
|
|
23
24
|
|
24
25
|
included do
|
25
26
|
# Default column to be used when creating Advisory Locks
|
26
|
-
|
27
|
+
class_attribute :advisory_lockable_column, instance_accessor: false, default: Concurrent::Delay.new { primary_key }
|
27
28
|
|
28
29
|
# Default Postgres function to be used for Advisory Locks
|
29
|
-
|
30
|
+
class_attribute :advisory_lockable_function, default: "pg_try_advisory_lock"
|
30
31
|
|
31
32
|
# Attempt to acquire an advisory lock on the selected records and
|
32
33
|
# return only those records for which a lock could be acquired.
|
33
|
-
# @!method advisory_lock(column:
|
34
|
+
# @!method advisory_lock(column: _advisory_lockable_column, function: advisory_lockable_function)
|
34
35
|
# @!scope class
|
35
36
|
# @param column [String, Symbol] column values to Advisory Lock against
|
36
37
|
# @param function [String, Symbol] Postgres Advisory Lock function name to use
|
37
38
|
# @return [ActiveRecord::Relation]
|
38
39
|
# A relation selecting only the records that were locked.
|
39
|
-
scope :advisory_lock, (lambda do |column:
|
40
|
+
scope :advisory_lock, (lambda do |column: _advisory_lockable_column, function: advisory_lockable_function|
|
40
41
|
original_query = self
|
41
42
|
|
42
43
|
cte_table = Arel::Table.new(:rows)
|
@@ -64,13 +65,13 @@ module GoodJob
|
|
64
65
|
#
|
65
66
|
# For details on +pg_locks+, see
|
66
67
|
# {https://www.postgresql.org/docs/current/view-pg-locks.html}.
|
67
|
-
# @!method joins_advisory_locks(column:
|
68
|
+
# @!method joins_advisory_locks(column: _advisory_lockable_column)
|
68
69
|
# @!scope class
|
69
70
|
# @param column [String, Symbol] column values to Advisory Lock against
|
70
71
|
# @return [ActiveRecord::Relation]
|
71
72
|
# @example Get the records that have a session awaiting a lock:
|
72
73
|
# MyLockableRecord.joins_advisory_locks.where("pg_locks.granted = ?", false)
|
73
|
-
scope :joins_advisory_locks, (lambda do |column:
|
74
|
+
scope :joins_advisory_locks, (lambda do |column: _advisory_lockable_column|
|
74
75
|
join_sql = <<~SQL.squish
|
75
76
|
LEFT JOIN pg_locks ON pg_locks.locktype = 'advisory'
|
76
77
|
AND pg_locks.objsubid = 1
|
@@ -82,26 +83,26 @@ module GoodJob
|
|
82
83
|
end)
|
83
84
|
|
84
85
|
# Find records that do not have an advisory lock on them.
|
85
|
-
# @!method advisory_unlocked(column:
|
86
|
+
# @!method advisory_unlocked(column: _advisory_lockable_column)
|
86
87
|
# @!scope class
|
87
88
|
# @param column [String, Symbol] column values to Advisory Lock against
|
88
89
|
# @return [ActiveRecord::Relation]
|
89
|
-
scope :advisory_unlocked, ->(column:
|
90
|
+
scope :advisory_unlocked, ->(column: _advisory_lockable_column) { joins_advisory_locks(column: column).where(pg_locks: { locktype: nil }) }
|
90
91
|
|
91
92
|
# Find records that have an advisory lock on them.
|
92
|
-
# @!method advisory_locked(column:
|
93
|
+
# @!method advisory_locked(column: _advisory_lockable_column)
|
93
94
|
# @!scope class
|
94
95
|
# @param column [String, Symbol] column values to Advisory Lock against
|
95
96
|
# @return [ActiveRecord::Relation]
|
96
|
-
scope :advisory_locked, ->(column:
|
97
|
+
scope :advisory_locked, ->(column: _advisory_lockable_column) { joins_advisory_locks(column: column).where.not(pg_locks: { locktype: nil }) }
|
97
98
|
|
98
99
|
# Find records with advisory locks owned by the current Postgres
|
99
100
|
# session/connection.
|
100
|
-
# @!method advisory_locked(column:
|
101
|
+
# @!method advisory_locked(column: _advisory_lockable_column)
|
101
102
|
# @!scope class
|
102
103
|
# @param column [String, Symbol] column values to Advisory Lock against
|
103
104
|
# @return [ActiveRecord::Relation]
|
104
|
-
scope :owns_advisory_locked, ->(column:
|
105
|
+
scope :owns_advisory_locked, ->(column: _advisory_lockable_column) { joins_advisory_locks(column: column).where('"pg_locks"."pid" = pg_backend_pid()') }
|
105
106
|
|
106
107
|
# Whether an advisory lock should be acquired in the same transaction
|
107
108
|
# that created the record.
|
@@ -143,7 +144,7 @@ module GoodJob
|
|
143
144
|
# MyLockableRecord.order(created_at: :asc).limit(2).with_advisory_lock do |record|
|
144
145
|
# do_something_with record
|
145
146
|
# end
|
146
|
-
def with_advisory_lock(column:
|
147
|
+
def with_advisory_lock(column: _advisory_lockable_column, function: advisory_lockable_function, unlock_session: false)
|
147
148
|
raise ArgumentError, "Must provide a block" unless block_given?
|
148
149
|
|
149
150
|
records = advisory_lock(column: column, function: function).to_a
|
@@ -154,13 +155,19 @@ module GoodJob
|
|
154
155
|
advisory_unlock_session
|
155
156
|
else
|
156
157
|
records.each do |record|
|
157
|
-
key = [table_name, record[
|
158
|
+
key = [table_name, record[_advisory_lockable_column]].join
|
158
159
|
record.advisory_unlock(key: key, function: advisory_unlockable_function(function))
|
159
160
|
end
|
160
161
|
end
|
161
162
|
end
|
162
163
|
end
|
163
164
|
|
165
|
+
# Allow advisory_lockable_column to be a `Concurrent::Delay`
|
166
|
+
def _advisory_lockable_column
|
167
|
+
column = advisory_lockable_column
|
168
|
+
column.respond_to?(:value) ? column.value : column
|
169
|
+
end
|
170
|
+
|
164
171
|
def supports_cte_materialization_specifiers?
|
165
172
|
return @_supports_cte_materialization_specifiers if defined?(@_supports_cte_materialization_specifiers)
|
166
173
|
|
@@ -201,9 +208,16 @@ module GoodJob
|
|
201
208
|
# @param function [String, Symbol] Postgres Advisory Lock function name to use
|
202
209
|
# @return [Boolean] whether the lock was acquired.
|
203
210
|
def advisory_lock(key: lockable_key, function: advisory_lockable_function)
|
204
|
-
query =
|
205
|
-
|
206
|
-
|
211
|
+
query = if function.include? "_try_"
|
212
|
+
<<~SQL.squish
|
213
|
+
SELECT #{function}(('x'||substr(md5($1::text), 1, 16))::bit(64)::bigint) AS locked
|
214
|
+
SQL
|
215
|
+
else
|
216
|
+
<<~SQL.squish
|
217
|
+
SELECT #{function}(('x'||substr(md5($1::text), 1, 16))::bit(64)::bigint)::text AS locked
|
218
|
+
SQL
|
219
|
+
end
|
220
|
+
|
207
221
|
binds = [[nil, key]]
|
208
222
|
self.class.connection.exec_query(pg_or_jdbc_query(query), 'GoodJob::Lockable Advisory Lock', binds).first['locked']
|
209
223
|
end
|
@@ -301,7 +315,7 @@ module GoodJob
|
|
301
315
|
# Default Advisory Lock key
|
302
316
|
# @return [String]
|
303
317
|
def lockable_key
|
304
|
-
[self.class.table_name, self[self.class.
|
318
|
+
[self.class.table_name, self[self.class._advisory_lockable_column]].join
|
305
319
|
end
|
306
320
|
|
307
321
|
delegate :pg_or_jdbc_query, to: :class
|
data/lib/good_job/notifier.rb
CHANGED
@@ -1,3 +1,4 @@
|
|
1
|
+
# frozen_string_literal: true
|
1
2
|
require 'concurrent/atomic/atomic_boolean'
|
2
3
|
|
3
4
|
module GoodJob # :nodoc:
|
@@ -13,7 +14,7 @@ module GoodJob # :nodoc:
|
|
13
14
|
AdapterCannotListenError = Class.new(StandardError)
|
14
15
|
|
15
16
|
# Default Postgres channel for LISTEN/NOTIFY
|
16
|
-
CHANNEL = 'good_job'
|
17
|
+
CHANNEL = 'good_job'
|
17
18
|
# Defaults for instance of Concurrent::ThreadPoolExecutor
|
18
19
|
EXECUTOR_OPTIONS = {
|
19
20
|
name: name,
|
@@ -24,6 +25,8 @@ module GoodJob # :nodoc:
|
|
24
25
|
max_queue: 1,
|
25
26
|
fallback_policy: :discard,
|
26
27
|
}.freeze
|
28
|
+
# Seconds to wait if database cannot be connected to
|
29
|
+
RECONNECT_INTERVAL = 5
|
27
30
|
# Seconds to block while LISTENing for a message
|
28
31
|
WAIT_INTERVAL = 1
|
29
32
|
|
@@ -114,7 +117,13 @@ module GoodJob # :nodoc:
|
|
114
117
|
ActiveSupport::Notifications.instrument("notifier_notify_error.good_job", { error: thread_error })
|
115
118
|
end
|
116
119
|
|
117
|
-
|
120
|
+
return if shutdown?
|
121
|
+
|
122
|
+
if thread_error.is_a?(ActiveRecord::ConnectionNotEstablished) || thread_error.is_a?(ActiveRecord::StatementInvalid)
|
123
|
+
listen(delay: RECONNECT_INTERVAL)
|
124
|
+
else
|
125
|
+
listen
|
126
|
+
end
|
118
127
|
end
|
119
128
|
|
120
129
|
private
|
@@ -125,8 +134,8 @@ module GoodJob # :nodoc:
|
|
125
134
|
@executor = Concurrent::ThreadPoolExecutor.new(EXECUTOR_OPTIONS)
|
126
135
|
end
|
127
136
|
|
128
|
-
def listen
|
129
|
-
future = Concurrent::
|
137
|
+
def listen(delay: 0)
|
138
|
+
future = Concurrent::ScheduledTask.new(delay, args: [@recipients, executor, @listening], executor: @executor) do |thr_recipients, thr_executor, thr_listening|
|
130
139
|
with_listen_connection do |conn|
|
131
140
|
ActiveSupport::Notifications.instrument("notifier_listen.good_job") do
|
132
141
|
conn.async_exec("LISTEN #{CHANNEL}").clear
|
data/lib/good_job/poller.rb
CHANGED
data/lib/good_job/railtie.rb
CHANGED
data/lib/good_job/scheduler.rb
CHANGED
data/lib/good_job/version.rb
CHANGED
metadata
CHANGED
@@ -1,14 +1,14 @@
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
2
2
|
name: good_job
|
3
3
|
version: !ruby/object:Gem::Version
|
4
|
-
version: 1.
|
4
|
+
version: 1.11.3
|
5
5
|
platform: ruby
|
6
6
|
authors:
|
7
7
|
- Ben Sheldon
|
8
8
|
autorequire:
|
9
9
|
bindir: exe
|
10
10
|
cert_chain: []
|
11
|
-
date: 2021-
|
11
|
+
date: 2021-07-25 00:00:00.000000000 Z
|
12
12
|
dependencies:
|
13
13
|
- !ruby/object:Gem::Dependency
|
14
14
|
name: activejob
|
@@ -362,6 +362,8 @@ files:
|
|
362
362
|
- lib/generators/good_job/templates/update/migrations/03_add_active_job_id_index_and_concurrency_key_index_to_good_jobs.rb
|
363
363
|
- lib/generators/good_job/update_generator.rb
|
364
364
|
- lib/good_job.rb
|
365
|
+
- lib/good_job/active_job_extensions.rb
|
366
|
+
- lib/good_job/active_job_extensions/concurrency.rb
|
365
367
|
- lib/good_job/adapter.rb
|
366
368
|
- lib/good_job/cli.rb
|
367
369
|
- lib/good_job/configuration.rb
|