good_job 1.9.5 → 1.11.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/CHANGELOG.md +81 -1
- data/README.md +47 -0
- data/engine/app/controllers/good_job/assets_controller.rb +1 -1
- data/engine/app/controllers/good_job/dashboards_controller.rb +1 -1
- data/engine/app/controllers/good_job/jobs_controller.rb +9 -0
- data/engine/app/views/layouts/good_job/base.html.erb +13 -0
- data/engine/app/views/shared/_jobs_table.erb +13 -1
- data/engine/app/views/shared/icons/_check.html.erb +4 -0
- data/engine/app/views/shared/icons/_exclamation.html.erb +4 -0
- data/engine/app/views/shared/icons/_trash.html.erb +5 -0
- data/engine/config/routes.rb +2 -1
- data/lib/generators/good_job/install_generator.rb +5 -15
- data/lib/generators/good_job/templates/install/migrations/create_good_jobs.rb.erb +27 -0
- data/lib/generators/good_job/templates/{migration.rb.erb → update/migrations/01_create_good_jobs.rb} +3 -3
- data/lib/generators/good_job/templates/update/migrations/02_add_active_job_id_concurrency_key_cron_key_to_good_jobs.rb +15 -0
- data/lib/generators/good_job/templates/update/migrations/03_add_active_job_id_index_and_concurrency_key_index_to_good_jobs.rb +32 -0
- data/lib/generators/good_job/update_generator.rb +29 -0
- data/lib/good_job/active_job_extensions.rb +4 -0
- data/lib/good_job/active_job_extensions/concurrency.rb +68 -0
- data/lib/good_job/current_execution.rb +12 -5
- data/lib/good_job/job.rb +43 -6
- data/lib/good_job/lockable.rb +120 -57
- data/lib/good_job/poller.rb +7 -3
- data/lib/good_job/version.rb +1 -1
- metadata +27 -3
checksums.yaml
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
---
|
2
2
|
SHA256:
|
3
|
-
metadata.gz:
|
4
|
-
data.tar.gz:
|
3
|
+
metadata.gz: 91a09f77d3cb46eb91c4673732e1c52ea752f8d94f95c23c81d3457dc63aa051
|
4
|
+
data.tar.gz: ef21168d4aa96f3dfbf50ebed2339132bea52542209d5732d84ac23028227537
|
5
5
|
SHA512:
|
6
|
-
metadata.gz:
|
7
|
-
data.tar.gz:
|
6
|
+
metadata.gz: 8140a558ee43eeb2e3673462b7279f25cbed256bac385b131aa48c2f0ff59106d2038a39c65f3f1fd11f65cbcf7d3fb232bbfd20c790e7de629047a8eaed742b
|
7
|
+
data.tar.gz: 9daf8d432f5a013fbe5af10176b788c6c2e8c89e0f9b5e12568cbef7a7ecddda2b9296968e1076f71e5f26ed35a2f37d987227e7ad260030807f95404c7dbdeb
|
data/CHANGELOG.md
CHANGED
@@ -1,9 +1,90 @@
|
|
1
1
|
# Changelog
|
2
2
|
|
3
|
+
## [v1.11.1](https://github.com/bensheldon/good_job/tree/v1.11.1) (2021-07-07)
|
4
|
+
|
5
|
+
[Full Changelog](https://github.com/bensheldon/good_job/compare/v1.11.0...v1.11.1)
|
6
|
+
|
7
|
+
**Closed issues:**
|
8
|
+
|
9
|
+
- Database connection required while loading the code on 1.10.x [\#291](https://github.com/bensheldon/good_job/issues/291)
|
10
|
+
|
11
|
+
**Merged pull requests:**
|
12
|
+
|
13
|
+
- Defer accessing ActiveRecord `primary_key` in Lockable [\#293](https://github.com/bensheldon/good_job/pull/293) ([bensheldon](https://github.com/bensheldon))
|
14
|
+
|
15
|
+
## [v1.11.0](https://github.com/bensheldon/good_job/tree/v1.11.0) (2021-07-07)
|
16
|
+
|
17
|
+
[Full Changelog](https://github.com/bensheldon/good_job/compare/v1.10.1...v1.11.0)
|
18
|
+
|
19
|
+
**Implemented enhancements:**
|
20
|
+
|
21
|
+
- Add concurrency extension for ActiveJob [\#281](https://github.com/bensheldon/good_job/pull/281) ([bensheldon](https://github.com/bensheldon))
|
22
|
+
|
23
|
+
**Closed issues:**
|
24
|
+
|
25
|
+
- Investigate GoodJob concurrency [\#289](https://github.com/bensheldon/good_job/issues/289)
|
26
|
+
- Problem with migrating database on 1.10.0 [\#287](https://github.com/bensheldon/good_job/issues/287)
|
27
|
+
- Support migration --database option for install task? [\#267](https://github.com/bensheldon/good_job/issues/267)
|
28
|
+
- Add GoodJob to Ruby Toolbox [\#243](https://github.com/bensheldon/good_job/issues/243)
|
29
|
+
- Custom advisory locks to prevent certain jobs from being worked on concurrently? [\#206](https://github.com/bensheldon/good_job/issues/206)
|
30
|
+
|
31
|
+
## [v1.10.1](https://github.com/bensheldon/good_job/tree/v1.10.1) (2021-06-30)
|
32
|
+
|
33
|
+
[Full Changelog](https://github.com/bensheldon/good_job/compare/v1.10.0...v1.10.1)
|
34
|
+
|
35
|
+
**Merged pull requests:**
|
36
|
+
|
37
|
+
- Remove `FOR UPDATE SKIP LOCKED` from job locking sql statement [\#288](https://github.com/bensheldon/good_job/pull/288) ([bensheldon](https://github.com/bensheldon))
|
38
|
+
- Update GH Test Matrix with latest JRuby 9.2.19.0 [\#283](https://github.com/bensheldon/good_job/pull/283) ([tedhexaflow](https://github.com/tedhexaflow))
|
39
|
+
|
40
|
+
## [v1.10.0](https://github.com/bensheldon/good_job/tree/v1.10.0) (2021-06-29)
|
41
|
+
|
42
|
+
[Full Changelog](https://github.com/bensheldon/good_job/compare/v1.9.6...v1.10.0)
|
43
|
+
|
44
|
+
**Implemented enhancements:**
|
45
|
+
|
46
|
+
- Add `rails g good_job:update` command to add idempotent migration files, including `active_job_id`, `concurrency_key`, `cron_key` columns [\#266](https://github.com/bensheldon/good_job/pull/266) ([bensheldon](https://github.com/bensheldon))
|
47
|
+
|
48
|
+
**Fixed bugs:**
|
49
|
+
|
50
|
+
- Use `pg_advisory_unlock_all` after each thread's job execution; fix Lockable return values; improve test stability [\#285](https://github.com/bensheldon/good_job/pull/285) ([bensheldon](https://github.com/bensheldon))
|
51
|
+
- Dashboard AssetsController does not raise if verify\_authenticity\_token is not in the callback chain [\#284](https://github.com/bensheldon/good_job/pull/284) ([bensheldon](https://github.com/bensheldon))
|
52
|
+
|
53
|
+
**Closed issues:**
|
54
|
+
|
55
|
+
- \[Question\] Dashboard assets not showing [\#282](https://github.com/bensheldon/good_job/issues/282)
|
56
|
+
|
57
|
+
**Merged pull requests:**
|
58
|
+
|
59
|
+
- Separately cache Appraisal gems in GH Action [\#280](https://github.com/bensheldon/good_job/pull/280) ([bensheldon](https://github.com/bensheldon))
|
60
|
+
- Use custom RSpec doc formatter to show spec examples that are running [\#279](https://github.com/bensheldon/good_job/pull/279) ([bensheldon](https://github.com/bensheldon))
|
61
|
+
- Update development dependencies [\#278](https://github.com/bensheldon/good_job/pull/278) ([bensheldon](https://github.com/bensheldon))
|
62
|
+
- Fix Scheduler integration spec to ensure jobs are run in the Scheduler under test [\#276](https://github.com/bensheldon/good_job/pull/276) ([bensheldon](https://github.com/bensheldon))
|
63
|
+
- Add example benchmark for job throughput [\#275](https://github.com/bensheldon/good_job/pull/275) ([bensheldon](https://github.com/bensheldon))
|
64
|
+
- Allow Lockable to be passed custom column, key, and Postgres advisory lock/unlock function [\#273](https://github.com/bensheldon/good_job/pull/273) ([bensheldon](https://github.com/bensheldon))
|
65
|
+
|
66
|
+
## [v1.9.6](https://github.com/bensheldon/good_job/tree/v1.9.6) (2021-06-04)
|
67
|
+
|
68
|
+
[Full Changelog](https://github.com/bensheldon/good_job/compare/v1.9.5...v1.9.6)
|
69
|
+
|
70
|
+
**Implemented enhancements:**
|
71
|
+
|
72
|
+
- Add deleting jobs from UI. [\#265](https://github.com/bensheldon/good_job/pull/265) ([morgoth](https://github.com/morgoth))
|
73
|
+
- Collapse Dashboard params by default [\#263](https://github.com/bensheldon/good_job/pull/263) ([morgoth](https://github.com/morgoth))
|
74
|
+
|
75
|
+
**Closed issues:**
|
76
|
+
|
77
|
+
- Pause jobs during migration / maintenance? [\#257](https://github.com/bensheldon/good_job/issues/257)
|
78
|
+
- How to properly report errors to error tracker service [\#159](https://github.com/bensheldon/good_job/issues/159)
|
79
|
+
|
3
80
|
## [v1.9.5](https://github.com/bensheldon/good_job/tree/v1.9.5) (2021-05-24)
|
4
81
|
|
5
82
|
[Full Changelog](https://github.com/bensheldon/good_job/compare/v1.9.4...v1.9.5)
|
6
83
|
|
84
|
+
**Implemented enhancements:**
|
85
|
+
|
86
|
+
- Update Dashboard to Bootstrap 5 [\#260](https://github.com/bensheldon/good_job/pull/260) ([morgoth](https://github.com/morgoth))
|
87
|
+
|
7
88
|
**Closed issues:**
|
8
89
|
|
9
90
|
- Update from bootstrap 4 to bootstrap 5 [\#258](https://github.com/bensheldon/good_job/issues/258)
|
@@ -12,7 +93,6 @@
|
|
12
93
|
|
13
94
|
- Serve Dashboard assets as discrete paths instead of inlining [\#262](https://github.com/bensheldon/good_job/pull/262) ([bensheldon](https://github.com/bensheldon))
|
14
95
|
- Fix Gemfile.lock's missing JRuby dependencies; fix release script and add check [\#261](https://github.com/bensheldon/good_job/pull/261) ([bensheldon](https://github.com/bensheldon))
|
15
|
-
- Update Dashboard to Bootstrap 5 [\#260](https://github.com/bensheldon/good_job/pull/260) ([morgoth](https://github.com/morgoth))
|
16
96
|
|
17
97
|
## [v1.9.4](https://github.com/bensheldon/good_job/tree/v1.9.4) (2021-05-18)
|
18
98
|
|
data/README.md
CHANGED
@@ -38,6 +38,8 @@ For more of the story of GoodJob, read the [introductory blog post](https://isla
|
|
38
38
|
- [Configuration options](#configuration-options)
|
39
39
|
- [Global options](#global-options)
|
40
40
|
- [Dashboard](#dashboard)
|
41
|
+
- [ActiveJob Concurrency](#activejob-concurrency)
|
42
|
+
- [Updating](#updating)
|
41
43
|
- [Go deeper](#go-deeper)
|
42
44
|
- [Exceptions, retries, and reliability](#exceptions-retries-and-reliability)
|
43
45
|
- [Exceptions](#exceptions)
|
@@ -318,6 +320,51 @@ GoodJob includes a Dashboard as a mountable `Rails::Engine`.
|
|
318
320
|
end
|
319
321
|
```
|
320
322
|
|
323
|
+
### ActiveJob Concurrency
|
324
|
+
|
325
|
+
GoodJob can extend ActiveJob to provide limits on concurrently running jobs, either at time of _enqueue_ or at _perform_.
|
326
|
+
|
327
|
+
**Note:** Limiting concurrency at _enqueue_ requires Rails 6.0+ because Rails 5.2 does not support `throw :abort` in ActiveJob callbacks.
|
328
|
+
|
329
|
+
```ruby
|
330
|
+
class MyJob < ApplicationJob
|
331
|
+
include GoodJob::ActiveJobExtensions::Concurrency
|
332
|
+
|
333
|
+
good_job_control_concurrency_with(
|
334
|
+
# Maximum number of jobs with the concurrency key to be concurrently enqueued
|
335
|
+
enqueue_limit: 2,
|
336
|
+
|
337
|
+
# Maximum number of jobs with the concurrency key to be concurrently performed
|
338
|
+
perform_limit: 1,
|
339
|
+
|
340
|
+
# A unique key to be globally locked against.
|
341
|
+
# Can be String or Lambda/Proc that is invoked in the context of the job.
|
342
|
+
# Note: Arguments passed to #perform_later must be accessed through `arguments` method.
|
343
|
+
key: -> { "Unique-#{arguments.first}" } # MyJob.perform_later("Alice") => "Unique-Alice"
|
344
|
+
)
|
345
|
+
|
346
|
+
def perform(first_name)
|
347
|
+
# do work
|
348
|
+
end
|
349
|
+
end
|
350
|
+
```
|
351
|
+
|
352
|
+
### Updating
|
353
|
+
|
354
|
+
GoodJob follows semantic versioning, though updates may be encouraged through deprecation warnings in minor versions.
|
355
|
+
|
356
|
+
To apply updates:
|
357
|
+
|
358
|
+
```bash
|
359
|
+
bin/rails g good_job:update
|
360
|
+
```
|
361
|
+
|
362
|
+
...and run the resulting migration:
|
363
|
+
|
364
|
+
```bash
|
365
|
+
bin/rails db:migrate
|
366
|
+
```
|
367
|
+
|
321
368
|
## Go deeper
|
322
369
|
|
323
370
|
### Exceptions, retries, and reliability
|
@@ -1,6 +1,6 @@
|
|
1
1
|
module GoodJob
|
2
2
|
class AssetsController < ActionController::Base # rubocop:disable Rails/ApplicationController
|
3
|
-
skip_before_action :verify_authenticity_token
|
3
|
+
skip_before_action :verify_authenticity_token, raise: false
|
4
4
|
|
5
5
|
before_action do
|
6
6
|
expires_in 1.year, public: true
|
@@ -14,7 +14,7 @@ module GoodJob
|
|
14
14
|
def jobs
|
15
15
|
after_scheduled_at = params[:after_scheduled_at].present? ? Time.zone.parse(params[:after_scheduled_at]) : nil
|
16
16
|
sql = GoodJob::Job.display_all(after_scheduled_at: after_scheduled_at, after_id: params[:after_id])
|
17
|
-
.limit(params.fetch(:limit,
|
17
|
+
.limit(params.fetch(:limit, 25))
|
18
18
|
sql = sql.with_job_class(params[:job_class]) if params[:job_class]
|
19
19
|
if params[:state]
|
20
20
|
case params[:state]
|
@@ -0,0 +1,9 @@
|
|
1
|
+
module GoodJob
|
2
|
+
class JobsController < GoodJob::BaseController
|
3
|
+
def destroy
|
4
|
+
deleted_count = GoodJob::Job.where(id: params[:id]).delete_all
|
5
|
+
message = deleted_count.positive? ? { notice: "Job deleted" } : { alert: "Job not deleted" }
|
6
|
+
redirect_to root_path, **message
|
7
|
+
end
|
8
|
+
end
|
9
|
+
end
|
@@ -51,6 +51,19 @@
|
|
51
51
|
</div>
|
52
52
|
</div>
|
53
53
|
|
54
|
+
<% if notice %>
|
55
|
+
<div class="alert alert-success alert-dismissible fade show d-flex align-items-center offset-md-3 col-6" role="alert">
|
56
|
+
<%= render "shared/icons/check", class: "flex-shrink-0 me-2" %>
|
57
|
+
<div><%= notice %></div>
|
58
|
+
<button type="button" class="btn-close" data-bs-dismiss="alert" aria-label="Close"></button>
|
59
|
+
</div>
|
60
|
+
<% elsif alert %>
|
61
|
+
<div class="alert alert-warning alert-dismissible fade show d-flex align-items-center offset-md-3 col-6" role="alert">
|
62
|
+
<%= render "shared/icons/check", class: "flex-shrink-0 me-2" %>
|
63
|
+
<div><%= alert %></div>
|
64
|
+
<button type="button" class="btn-close" data-bs-dismiss="alert" aria-label="Close"></button>
|
65
|
+
</div>
|
66
|
+
<% end %>
|
54
67
|
<%= yield %>
|
55
68
|
</div>
|
56
69
|
</body>
|
@@ -9,6 +9,7 @@
|
|
9
9
|
<th>Scheduled At</th>
|
10
10
|
<th>Error</th>
|
11
11
|
<th>ActiveJob Params</th>
|
12
|
+
<th>Actions</th>
|
12
13
|
</thead>
|
13
14
|
<tbody>
|
14
15
|
<% jobs.each do |job| %>
|
@@ -19,7 +20,18 @@
|
|
19
20
|
<td><%= job.queue_name %></td>
|
20
21
|
<td><%= job.scheduled_at || job.created_at %></td>
|
21
22
|
<td><%= job.error %></td>
|
22
|
-
<td
|
23
|
+
<td>
|
24
|
+
<%= tag.button "Preview", type: "button", class: "btn btn-sm btn-outline-primary", role: "button",
|
25
|
+
data: {bs_toggle: "collapse", bs_target: "##{dom_id(job, 'params')}"},
|
26
|
+
aria: {expanded: false, controls: dom_id(job, "params")}
|
27
|
+
%>
|
28
|
+
<%= tag.pre JSON.pretty_generate(job.serialized_params), id: dom_id(job, "params"), class: "collapse" %>
|
29
|
+
</td>
|
30
|
+
<td>
|
31
|
+
<%= button_to job_path(job.id), method: :delete, class: "btn btn-sm btn-outline-danger", title: "Delete job" do %>
|
32
|
+
<%= render "shared/icons/trash" %>
|
33
|
+
<% end %>
|
34
|
+
</td>
|
23
35
|
</tr>
|
24
36
|
<% end %>
|
25
37
|
</tbody>
|
@@ -0,0 +1,4 @@
|
|
1
|
+
<!-- https://icons.getbootstrap.com/icons/check-circle-fill/ -->
|
2
|
+
<svg xmlns="http://www.w3.org/2000/svg" width="16" height="16" fill="currentColor" class="bi bi-check-circle-fill <%= local_assigns[:class] %>" viewBox="0 0 16 16">
|
3
|
+
<path d="M16 8A8 8 0 1 1 0 8a8 8 0 0 1 16 0zm-3.97-3.03a.75.75 0 0 0-1.08.022L7.477 9.417 5.384 7.323a.75.75 0 0 0-1.06 1.06L6.97 11.03a.75.75 0 0 0 1.079-.02l3.992-4.99a.75.75 0 0 0-.01-1.05z" />
|
4
|
+
</svg>
|
@@ -0,0 +1,4 @@
|
|
1
|
+
<!-- https://icons.getbootstrap.com/icons/exclamation-triangle-fill/ -->
|
2
|
+
<svg xmlns="http://www.w3.org/2000/svg" width="16" height="16" fill="currentColor" class="bi bi-exclamation-triangle-fill <%= local_assigns[:class] %>" viewBox="0 0 16 16">
|
3
|
+
<path d="M8.982 1.566a1.13 1.13 0 0 0-1.96 0L.165 13.233c-.457.778.091 1.767.98 1.767h13.713c.889 0 1.438-.99.98-1.767L8.982 1.566zM8 5c.535 0 .954.462.9.995l-.35 3.507a.552.552 0 0 1-1.1 0L7.1 5.995A.905.905 0 0 1 8 5zm.002 6a1 1 0 1 1 0 2 1 1 0 0 1 0-2z" />
|
4
|
+
</svg>
|
@@ -0,0 +1,5 @@
|
|
1
|
+
<!-- https://icons.getbootstrap.com/icons/trash/ -->
|
2
|
+
<svg xmlns="http://www.w3.org/2000/svg" width="16" height="16" fill="currentColor" class="bi bi-trash" viewBox="0 0 16 16">
|
3
|
+
<path d="M5.5 5.5A.5.5 0 0 1 6 6v6a.5.5 0 0 1-1 0V6a.5.5 0 0 1 .5-.5zm2.5 0a.5.5 0 0 1 .5.5v6a.5.5 0 0 1-1 0V6a.5.5 0 0 1 .5-.5zm3 .5a.5.5 0 0 0-1 0v6a.5.5 0 0 0 1 0V6z" />
|
4
|
+
<path fill-rule="evenodd" d="M14.5 3a1 1 0 0 1-1 1H13v9a2 2 0 0 1-2 2H5a2 2 0 0 1-2-2V4h-.5a1 1 0 0 1-1-1V2a1 1 0 0 1 1-1H6a1 1 0 0 1 1-1h2a1 1 0 0 1 1 1h3.5a1 1 0 0 1 1 1v1zM4.118 4 4 4.059V13a1 1 0 0 0 1 1h6a1 1 0 0 0 1-1V4.059L11.882 4H4.118zM2.5 3V2h11v1h-11z" />
|
5
|
+
</svg>
|
data/engine/config/routes.rb
CHANGED
@@ -1,13 +1,9 @@
|
|
1
1
|
require 'rails/generators'
|
2
2
|
require 'rails/generators/active_record'
|
3
|
-
|
4
3
|
module GoodJob
|
5
4
|
#
|
6
|
-
#
|
7
|
-
#
|
8
|
-
#
|
9
|
-
# This generator is primarily dedicated to stubbing out a migration that adds
|
10
|
-
# a table to hold GoodJob's queued jobs in your database.
|
5
|
+
# Rails generator used for setting up GoodJob in a Rails application.
|
6
|
+
# Run it with +bin/rails g good_job:install+ in your console.
|
11
7
|
#
|
12
8
|
class InstallGenerator < Rails::Generators::Base
|
13
9
|
include Rails::Generators::Migration
|
@@ -16,17 +12,11 @@ module GoodJob
|
|
16
12
|
delegate :next_migration_number, to: ActiveRecord::Generators::Base
|
17
13
|
end
|
18
14
|
|
19
|
-
source_paths << File.join(File.dirname(__FILE__), "templates")
|
15
|
+
source_paths << File.join(File.dirname(__FILE__), "templates/install")
|
20
16
|
|
21
|
-
# Generates
|
17
|
+
# Generates monolithic migration file that contains all database changes.
|
22
18
|
def create_migration_file
|
23
|
-
migration_template '
|
24
|
-
end
|
25
|
-
|
26
|
-
private
|
27
|
-
|
28
|
-
def migration_version
|
29
|
-
"[#{ActiveRecord::VERSION::MAJOR}.#{ActiveRecord::VERSION::MINOR}]"
|
19
|
+
migration_template 'migrations/create_good_jobs.rb.erb', 'db/migrate/create_good_jobs.rb'
|
30
20
|
end
|
31
21
|
end
|
32
22
|
end
|
@@ -0,0 +1,27 @@
|
|
1
|
+
class CreateGoodJobs < ActiveRecord::Migration[5.2]
|
2
|
+
def change
|
3
|
+
enable_extension 'pgcrypto'
|
4
|
+
|
5
|
+
create_table :good_jobs, id: :uuid do |t|
|
6
|
+
t.text :queue_name
|
7
|
+
t.integer :priority
|
8
|
+
t.jsonb :serialized_params
|
9
|
+
t.timestamp :scheduled_at
|
10
|
+
t.timestamp :performed_at
|
11
|
+
t.timestamp :finished_at
|
12
|
+
t.text :error
|
13
|
+
|
14
|
+
t.timestamps
|
15
|
+
|
16
|
+
t.uuid :active_job_id
|
17
|
+
t.text :concurrency_key
|
18
|
+
t.text :cron_key
|
19
|
+
end
|
20
|
+
|
21
|
+
add_index :good_jobs, :scheduled_at, where: "(finished_at IS NULL)", name: "index_good_jobs_on_scheduled_at"
|
22
|
+
add_index :good_jobs, [:queue_name, :scheduled_at], where: "(finished_at IS NULL)", name: :index_good_jobs_on_queue_name_and_scheduled_at
|
23
|
+
add_index :good_jobs, [:active_job_id, :created_at], name: :index_good_jobs_on_active_job_id_and_created_at
|
24
|
+
add_index :good_jobs, :concurrency_key, where: "(finished_at IS NULL)", name: :index_good_jobs_on_concurrency_key_when_unfinished
|
25
|
+
add_index :good_jobs, [:cron_key, :created_at], name: :index_good_jobs_on_cron_key_and_created_at
|
26
|
+
end
|
27
|
+
end
|
data/lib/generators/good_job/templates/{migration.rb.erb → update/migrations/01_create_good_jobs.rb}
RENAMED
@@ -1,4 +1,4 @@
|
|
1
|
-
class CreateGoodJobs < ActiveRecord::Migration
|
1
|
+
class CreateGoodJobs < ActiveRecord::Migration[5.2]
|
2
2
|
def change
|
3
3
|
enable_extension 'pgcrypto'
|
4
4
|
|
@@ -14,7 +14,7 @@ class CreateGoodJobs < ActiveRecord::Migration<%= migration_version %>
|
|
14
14
|
t.timestamps
|
15
15
|
end
|
16
16
|
|
17
|
-
add_index :good_jobs, :scheduled_at, where: "(finished_at IS NULL)"
|
18
|
-
add_index :good_jobs, [:queue_name, :scheduled_at], where: "(finished_at IS NULL)"
|
17
|
+
add_index :good_jobs, :scheduled_at, where: "(finished_at IS NULL)", name: "index_good_jobs_on_scheduled_at"
|
18
|
+
add_index :good_jobs, [:queue_name, :scheduled_at], where: "(finished_at IS NULL)", name: :index_good_jobs_on_queue_name_and_scheduled_at
|
19
19
|
end
|
20
20
|
end
|
@@ -0,0 +1,15 @@
|
|
1
|
+
class AddActiveJobIdConcurrencyKeyCronKeyToGoodJobs < ActiveRecord::Migration[5.2]
|
2
|
+
def change
|
3
|
+
reversible do |dir|
|
4
|
+
dir.up do
|
5
|
+
# Ensure this incremental update migration is idempotent
|
6
|
+
# with monolithic install migration.
|
7
|
+
return if connection.column_exists?(:good_jobs, :active_job_id)
|
8
|
+
end
|
9
|
+
end
|
10
|
+
|
11
|
+
add_column :good_jobs, :active_job_id, :uuid
|
12
|
+
add_column :good_jobs, :concurrency_key, :text
|
13
|
+
add_column :good_jobs, :cron_key, :text
|
14
|
+
end
|
15
|
+
end
|
@@ -0,0 +1,32 @@
|
|
1
|
+
class AddActiveJobIdIndexAndConcurrencyKeyIndexToGoodJobs < ActiveRecord::Migration[5.2]
|
2
|
+
disable_ddl_transaction!
|
3
|
+
|
4
|
+
UPDATE_BATCH_SIZE = 1_000
|
5
|
+
|
6
|
+
class GoodJobJobs < ActiveRecord::Base
|
7
|
+
self.table_name = "good_jobs"
|
8
|
+
end
|
9
|
+
|
10
|
+
def change
|
11
|
+
reversible do |dir|
|
12
|
+
dir.up do
|
13
|
+
# Ensure this incremental update migration is idempotent
|
14
|
+
# with monolithic install migration.
|
15
|
+
return if connection.index_name_exists?(:good_jobs, :index_good_jobs_on_active_job_id_and_created_at)
|
16
|
+
end
|
17
|
+
end
|
18
|
+
|
19
|
+
add_index :good_jobs, [:active_job_id, :created_at], algorithm: :concurrently, name: :index_good_jobs_on_active_job_id_and_created_at
|
20
|
+
add_index :good_jobs, :concurrency_key, where: "(finished_at IS NULL)", algorithm: :concurrently, name: :index_good_jobs_on_concurrency_key_when_unfinished
|
21
|
+
add_index :good_jobs, [:cron_key, :created_at], algorithm: :concurrently, name: :index_good_jobs_on_cron_key_and_created_at
|
22
|
+
|
23
|
+
reversible do |dir|
|
24
|
+
dir.up do
|
25
|
+
start_time = Time.current
|
26
|
+
loop do
|
27
|
+
break if GoodJobJobs.where(active_job_id: nil, finished_at: nil).where("created_at < ?", start_time).limit(UPDATE_BATCH_SIZE).update_all("active_job_id = (serialized_params->>'job_id')::uuid").zero?
|
28
|
+
end
|
29
|
+
end
|
30
|
+
end
|
31
|
+
end
|
32
|
+
end
|
@@ -0,0 +1,29 @@
|
|
1
|
+
require 'rails/generators'
|
2
|
+
require 'rails/generators/active_record'
|
3
|
+
|
4
|
+
module GoodJob
|
5
|
+
#
|
6
|
+
# Rails generator used for updating GoodJob in a Rails application.
|
7
|
+
# Run it with +bin/rails g good_job:update+ in your console.
|
8
|
+
#
|
9
|
+
class UpdateGenerator < Rails::Generators::Base
|
10
|
+
include Rails::Generators::Migration
|
11
|
+
|
12
|
+
class << self
|
13
|
+
delegate :next_migration_number, to: ActiveRecord::Generators::Base
|
14
|
+
end
|
15
|
+
|
16
|
+
TEMPLATES = File.join(File.dirname(__FILE__), "templates/update")
|
17
|
+
source_paths << TEMPLATES
|
18
|
+
|
19
|
+
# Generates incremental migration files unless they already exist.
|
20
|
+
# All migrations should be idempotent e.g. +add_index+ is guarded with +if_index_exists?+
|
21
|
+
def update_migration_files
|
22
|
+
migration_templates = Dir.children(File.join(TEMPLATES, 'migrations')).sort
|
23
|
+
migration_templates.each do |template_file|
|
24
|
+
destination_file = template_file.match(/^\d*_(.*\.rb)/)[1] # 01_create_good_jobs.rb.erb => create_good_jobs.rb
|
25
|
+
migration_template "migrations/#{template_file}", "db/migrate/#{destination_file}", skip: true
|
26
|
+
end
|
27
|
+
end
|
28
|
+
end
|
29
|
+
end
|
@@ -0,0 +1,68 @@
|
|
1
|
+
module GoodJob
|
2
|
+
module ActiveJobExtensions
|
3
|
+
module Concurrency
|
4
|
+
extend ActiveSupport::Concern
|
5
|
+
|
6
|
+
ConcurrencyExceededError = Class.new(StandardError)
|
7
|
+
|
8
|
+
included do
|
9
|
+
class_attribute :good_job_concurrency_config, instance_accessor: false, default: {}
|
10
|
+
|
11
|
+
before_enqueue do |job|
|
12
|
+
# Always allow jobs to be retried because the current job's execution will complete momentarily
|
13
|
+
next if CurrentExecution.active_job_id == job.job_id
|
14
|
+
|
15
|
+
limit = job.class.good_job_concurrency_config.fetch(:enqueue_limit, Float::INFINITY)
|
16
|
+
next if limit.blank? || (0...Float::INFINITY).exclude?(limit)
|
17
|
+
|
18
|
+
key = job.good_job_concurrency_key
|
19
|
+
next if key.blank?
|
20
|
+
|
21
|
+
GoodJob::Job.new.with_advisory_lock(key: key, function: "pg_advisory_lock") do
|
22
|
+
# TODO: Why is `unscoped` necessary? Nested scope is bleeding into subsequent query?
|
23
|
+
enqueue_concurrency = GoodJob::Job.unscoped.where(concurrency_key: key).unfinished.count
|
24
|
+
# The job has not yet been enqueued, so check if adding it will go over the limit
|
25
|
+
throw :abort if enqueue_concurrency + 1 > limit
|
26
|
+
end
|
27
|
+
end
|
28
|
+
|
29
|
+
retry_on(
|
30
|
+
GoodJob::ActiveJobExtensions::Concurrency::ConcurrencyExceededError,
|
31
|
+
attempts: Float::INFINITY,
|
32
|
+
wait: :exponentially_longer
|
33
|
+
)
|
34
|
+
|
35
|
+
before_perform do |job|
|
36
|
+
limit = job.class.good_job_concurrency_config.fetch(:perform_limit, Float::INFINITY)
|
37
|
+
next if limit.blank? || (0...Float::INFINITY).exclude?(limit)
|
38
|
+
|
39
|
+
key = job.good_job_concurrency_key
|
40
|
+
next if key.blank?
|
41
|
+
|
42
|
+
GoodJob::Job.new.with_advisory_lock(key: key, function: "pg_advisory_lock") do
|
43
|
+
perform_concurrency = GoodJob::Job.unscoped.where(concurrency_key: key).advisory_locked.count
|
44
|
+
# The current job has already been locked and will appear in the previous query
|
45
|
+
raise GoodJob::ActiveJobExtensions::Concurrency::ConcurrencyExceededError if perform_concurrency > limit
|
46
|
+
end
|
47
|
+
end
|
48
|
+
end
|
49
|
+
|
50
|
+
class_methods do
|
51
|
+
def good_job_control_concurrency_with(config)
|
52
|
+
self.good_job_concurrency_config = config
|
53
|
+
end
|
54
|
+
end
|
55
|
+
|
56
|
+
def good_job_concurrency_key
|
57
|
+
key = self.class.good_job_concurrency_config[:key]
|
58
|
+
return if key.blank?
|
59
|
+
|
60
|
+
if key.respond_to? :call
|
61
|
+
instance_exec(&key)
|
62
|
+
else
|
63
|
+
key
|
64
|
+
end
|
65
|
+
end
|
66
|
+
end
|
67
|
+
end
|
68
|
+
end
|
@@ -4,11 +4,11 @@ module GoodJob
|
|
4
4
|
# Thread-local attributes for passing values from Instrumentation.
|
5
5
|
# (Cannot use ActiveSupport::CurrentAttributes because ActiveJob resets it)
|
6
6
|
module CurrentExecution
|
7
|
-
# @!attribute [rw]
|
7
|
+
# @!attribute [rw] active_job_id
|
8
8
|
# @!scope class
|
9
|
-
#
|
10
|
-
# @return [
|
11
|
-
thread_mattr_accessor :
|
9
|
+
# ActiveJob ID
|
10
|
+
# @return [String, nil]
|
11
|
+
thread_mattr_accessor :active_job_id
|
12
12
|
|
13
13
|
# @!attribute [rw] error_on_discard
|
14
14
|
# @!scope class
|
@@ -16,11 +16,18 @@ module GoodJob
|
|
16
16
|
# @return [Exception, nil]
|
17
17
|
thread_mattr_accessor :error_on_discard
|
18
18
|
|
19
|
+
# @!attribute [rw] error_on_retry
|
20
|
+
# @!scope class
|
21
|
+
# Error captured by retry_on
|
22
|
+
# @return [Exception, nil]
|
23
|
+
thread_mattr_accessor :error_on_retry
|
24
|
+
|
19
25
|
# Resets attributes
|
20
26
|
# @return [void]
|
21
27
|
def self.reset
|
22
|
-
self.
|
28
|
+
self.active_job_id = nil
|
23
29
|
self.error_on_discard = nil
|
30
|
+
self.error_on_retry = nil
|
24
31
|
end
|
25
32
|
|
26
33
|
# @return [Integer] Current process ID
|
data/lib/good_job/job.rb
CHANGED
@@ -156,10 +156,10 @@ module GoodJob
|
|
156
156
|
# raised, if any (if the job raised, then the second array entry will be
|
157
157
|
# +nil+). If there were no jobs to execute, returns +nil+.
|
158
158
|
def self.perform_with_advisory_lock
|
159
|
-
unfinished.priority_ordered.only_scheduled.limit(1).with_advisory_lock do |good_jobs|
|
159
|
+
unfinished.priority_ordered.only_scheduled.limit(1).with_advisory_lock(unlock_session: true) do |good_jobs|
|
160
160
|
good_job = good_jobs.first
|
161
|
-
|
162
|
-
break unless good_job&.executable?
|
161
|
+
break if good_job.blank?
|
162
|
+
break :unlocked unless good_job&.executable?
|
163
163
|
|
164
164
|
good_job.perform
|
165
165
|
end
|
@@ -196,13 +196,45 @@ module GoodJob
|
|
196
196
|
# The new {Job} instance representing the queued ActiveJob job.
|
197
197
|
def self.enqueue(active_job, scheduled_at: nil, create_with_advisory_lock: false)
|
198
198
|
ActiveSupport::Notifications.instrument("enqueue_job.good_job", { active_job: active_job, scheduled_at: scheduled_at, create_with_advisory_lock: create_with_advisory_lock }) do |instrument_payload|
|
199
|
-
|
199
|
+
good_job_args = {
|
200
200
|
queue_name: active_job.queue_name.presence || DEFAULT_QUEUE_NAME,
|
201
201
|
priority: active_job.priority || DEFAULT_PRIORITY,
|
202
202
|
serialized_params: active_job.serialize,
|
203
203
|
scheduled_at: scheduled_at,
|
204
|
-
create_with_advisory_lock: create_with_advisory_lock
|
205
|
-
|
204
|
+
create_with_advisory_lock: create_with_advisory_lock,
|
205
|
+
}
|
206
|
+
|
207
|
+
if column_names.include?('active_job_id')
|
208
|
+
good_job_args[:active_job_id] = active_job.job_id
|
209
|
+
else
|
210
|
+
ActiveSupport::Deprecation.warn(<<~DEPRECATION)
|
211
|
+
GoodJob has pending database migrations. To create the migration files, run:
|
212
|
+
|
213
|
+
rails generate good_job:update
|
214
|
+
|
215
|
+
To apply the migration files, run:
|
216
|
+
|
217
|
+
rails db:migrate
|
218
|
+
|
219
|
+
DEPRECATION
|
220
|
+
end
|
221
|
+
|
222
|
+
if column_names.include?('concurrency_key')
|
223
|
+
good_job_args[:concurrency_key] = active_job.good_job_concurrency_key if active_job.respond_to?(:good_job_concurrency_key)
|
224
|
+
else
|
225
|
+
ActiveSupport::Deprecation.warn(<<~DEPRECATION)
|
226
|
+
GoodJob has pending database migrations. To create the migration files, run:
|
227
|
+
|
228
|
+
rails generate good_job:update
|
229
|
+
|
230
|
+
To apply the migration files, run:
|
231
|
+
|
232
|
+
rails db:migrate
|
233
|
+
|
234
|
+
DEPRECATION
|
235
|
+
end
|
236
|
+
|
237
|
+
good_job = GoodJob::Job.new(**good_job_args)
|
206
238
|
|
207
239
|
instrument_payload[:good_job] = good_job
|
208
240
|
|
@@ -247,6 +279,10 @@ module GoodJob
|
|
247
279
|
self.class.unscoped.unfinished.owns_advisory_locked.exists?(id: id)
|
248
280
|
end
|
249
281
|
|
282
|
+
def active_job_id
|
283
|
+
super || serialized_params['job_id']
|
284
|
+
end
|
285
|
+
|
250
286
|
private
|
251
287
|
|
252
288
|
# @return [ExecutionResult]
|
@@ -256,6 +292,7 @@ module GoodJob
|
|
256
292
|
)
|
257
293
|
|
258
294
|
GoodJob::CurrentExecution.reset
|
295
|
+
GoodJob::CurrentExecution.active_job_id = active_job_id
|
259
296
|
ActiveSupport::Notifications.instrument("perform_job.good_job", { good_job: self, process_id: GoodJob::CurrentExecution.process_id, thread_name: GoodJob::CurrentExecution.thread_name }) do
|
260
297
|
value = ActiveJob::Base.execute(params)
|
261
298
|
|
data/lib/good_job/lockable.rb
CHANGED
@@ -22,17 +22,25 @@ module GoodJob
|
|
22
22
|
RecordAlreadyAdvisoryLockedError = Class.new(StandardError)
|
23
23
|
|
24
24
|
included do
|
25
|
+
# Default column to be used when creating Advisory Locks
|
26
|
+
class_attribute :advisory_lockable_column, instance_accessor: false, default: Concurrent::Delay.new { primary_key }
|
27
|
+
|
28
|
+
# Default Postgres function to be used for Advisory Locks
|
29
|
+
class_attribute :advisory_lockable_function, default: "pg_try_advisory_lock"
|
30
|
+
|
25
31
|
# Attempt to acquire an advisory lock on the selected records and
|
26
32
|
# return only those records for which a lock could be acquired.
|
27
|
-
# @!method advisory_lock
|
33
|
+
# @!method advisory_lock(column: _advisory_lockable_column, function: advisory_lockable_function)
|
28
34
|
# @!scope class
|
35
|
+
# @param column [String, Symbol] column values to Advisory Lock against
|
36
|
+
# @param function [String, Symbol] Postgres Advisory Lock function name to use
|
29
37
|
# @return [ActiveRecord::Relation]
|
30
38
|
# A relation selecting only the records that were locked.
|
31
|
-
scope :advisory_lock, (lambda do
|
39
|
+
scope :advisory_lock, (lambda do |column: _advisory_lockable_column, function: advisory_lockable_function|
|
32
40
|
original_query = self
|
33
41
|
|
34
42
|
cte_table = Arel::Table.new(:rows)
|
35
|
-
cte_query = original_query.select(primary_key).except(:limit)
|
43
|
+
cte_query = original_query.select(primary_key, column).except(:limit)
|
36
44
|
cte_type = if supports_cte_materialization_specifiers?
|
37
45
|
'MATERIALIZED'
|
38
46
|
else
|
@@ -40,10 +48,9 @@ module GoodJob
|
|
40
48
|
end
|
41
49
|
|
42
50
|
composed_cte = Arel::Nodes::As.new(cte_table, Arel::Nodes::SqlLiteral.new([cte_type, "(", cte_query.to_sql, ")"].join(' ')))
|
43
|
-
|
44
51
|
query = cte_table.project(cte_table[:id])
|
45
52
|
.with(composed_cte)
|
46
|
-
.where(Arel.sql(sanitize_sql_for_conditions(["
|
53
|
+
.where(Arel.sql(sanitize_sql_for_conditions(["#{function}(('x' || substr(md5(:table_name || #{connection.quote_table_name(cte_table.name)}.#{connection.quote_column_name(column)}::text), 1, 16))::bit(64)::bigint)", { table_name: table_name }])))
|
47
54
|
|
48
55
|
limit = original_query.arel.ast.limit
|
49
56
|
query.limit = limit.value if limit.present?
|
@@ -57,40 +64,44 @@ module GoodJob
|
|
57
64
|
#
|
58
65
|
# For details on +pg_locks+, see
|
59
66
|
# {https://www.postgresql.org/docs/current/view-pg-locks.html}.
|
60
|
-
# @!method joins_advisory_locks
|
67
|
+
# @!method joins_advisory_locks(column: _advisory_lockable_column)
|
61
68
|
# @!scope class
|
69
|
+
# @param column [String, Symbol] column values to Advisory Lock against
|
62
70
|
# @return [ActiveRecord::Relation]
|
63
71
|
# @example Get the records that have a session awaiting a lock:
|
64
72
|
# MyLockableRecord.joins_advisory_locks.where("pg_locks.granted = ?", false)
|
65
|
-
scope :joins_advisory_locks, (lambda do
|
73
|
+
scope :joins_advisory_locks, (lambda do |column: _advisory_lockable_column|
|
66
74
|
join_sql = <<~SQL.squish
|
67
75
|
LEFT JOIN pg_locks ON pg_locks.locktype = 'advisory'
|
68
76
|
AND pg_locks.objsubid = 1
|
69
|
-
AND pg_locks.classid = ('x' || substr(md5(:table_name || #{quoted_table_name}.#{
|
70
|
-
AND pg_locks.objid = (('x' || substr(md5(:table_name || #{quoted_table_name}.#{
|
77
|
+
AND pg_locks.classid = ('x' || substr(md5(:table_name || #{quoted_table_name}.#{connection.quote_column_name(column)}::text), 1, 16))::bit(32)::int
|
78
|
+
AND pg_locks.objid = (('x' || substr(md5(:table_name || #{quoted_table_name}.#{connection.quote_column_name(column)}::text), 1, 16))::bit(64) << 32)::bit(32)::int
|
71
79
|
SQL
|
72
80
|
|
73
81
|
joins(sanitize_sql_for_conditions([join_sql, { table_name: table_name }]))
|
74
82
|
end)
|
75
83
|
|
76
84
|
# Find records that do not have an advisory lock on them.
|
77
|
-
# @!method advisory_unlocked
|
85
|
+
# @!method advisory_unlocked(column: _advisory_lockable_column)
|
78
86
|
# @!scope class
|
87
|
+
# @param column [String, Symbol] column values to Advisory Lock against
|
79
88
|
# @return [ActiveRecord::Relation]
|
80
|
-
scope :advisory_unlocked, -> { joins_advisory_locks.where(pg_locks: { locktype: nil }) }
|
89
|
+
scope :advisory_unlocked, ->(column: _advisory_lockable_column) { joins_advisory_locks(column: column).where(pg_locks: { locktype: nil }) }
|
81
90
|
|
82
91
|
# Find records that have an advisory lock on them.
|
83
|
-
# @!method advisory_locked
|
92
|
+
# @!method advisory_locked(column: _advisory_lockable_column)
|
84
93
|
# @!scope class
|
94
|
+
# @param column [String, Symbol] column values to Advisory Lock against
|
85
95
|
# @return [ActiveRecord::Relation]
|
86
|
-
scope :advisory_locked, -> { joins_advisory_locks.where.not(pg_locks: { locktype: nil }) }
|
96
|
+
scope :advisory_locked, ->(column: _advisory_lockable_column) { joins_advisory_locks(column: column).where.not(pg_locks: { locktype: nil }) }
|
87
97
|
|
88
98
|
# Find records with advisory locks owned by the current Postgres
|
89
99
|
# session/connection.
|
90
|
-
# @!method advisory_locked
|
100
|
+
# @!method advisory_locked(column: _advisory_lockable_column)
|
91
101
|
# @!scope class
|
102
|
+
# @param column [String, Symbol] column values to Advisory Lock against
|
92
103
|
# @return [ActiveRecord::Relation]
|
93
|
-
scope :owns_advisory_locked, -> { joins_advisory_locks.where('"pg_locks"."pid" = pg_backend_pid()') }
|
104
|
+
scope :owns_advisory_locked, ->(column: _advisory_lockable_column) { joins_advisory_locks(column: column).where('"pg_locks"."pid" = pg_backend_pid()') }
|
94
105
|
|
95
106
|
# Whether an advisory lock should be acquired in the same transaction
|
96
107
|
# that created the record.
|
@@ -122,6 +133,9 @@ module GoodJob
|
|
122
133
|
# can (as in {Lockable.advisory_lock}) and only pass those that could be
|
123
134
|
# locked to the block.
|
124
135
|
#
|
136
|
+
# @param column [String, Symbol] name of advisory lock or unlock function
|
137
|
+
# @param function [String, Symbol] Postgres Advisory Lock function name to use
|
138
|
+
# @param unlock_session [Boolean] Whether to unlock all advisory locks in the session afterwards
|
125
139
|
# @yield [Array<Lockable>] the records that were successfully locked.
|
126
140
|
# @return [Object] the result of the block.
|
127
141
|
#
|
@@ -129,65 +143,115 @@ module GoodJob
|
|
129
143
|
# MyLockableRecord.order(created_at: :asc).limit(2).with_advisory_lock do |record|
|
130
144
|
# do_something_with record
|
131
145
|
# end
|
132
|
-
def with_advisory_lock
|
146
|
+
def with_advisory_lock(column: _advisory_lockable_column, function: advisory_lockable_function, unlock_session: false)
|
133
147
|
raise ArgumentError, "Must provide a block" unless block_given?
|
134
148
|
|
135
|
-
records = advisory_lock.to_a
|
149
|
+
records = advisory_lock(column: column, function: function).to_a
|
136
150
|
begin
|
137
151
|
yield(records)
|
138
152
|
ensure
|
139
|
-
|
153
|
+
if unlock_session
|
154
|
+
advisory_unlock_session
|
155
|
+
else
|
156
|
+
records.each do |record|
|
157
|
+
key = [table_name, record[_advisory_lockable_column]].join
|
158
|
+
record.advisory_unlock(key: key, function: advisory_unlockable_function(function))
|
159
|
+
end
|
160
|
+
end
|
140
161
|
end
|
141
162
|
end
|
142
163
|
|
164
|
+
# Allow advisory_lockable_column to be a `Concurrent::Delay`
|
165
|
+
def _advisory_lockable_column
|
166
|
+
column = advisory_lockable_column
|
167
|
+
column.respond_to?(:value) ? column.value : column
|
168
|
+
end
|
169
|
+
|
143
170
|
def supports_cte_materialization_specifiers?
|
144
171
|
return @_supports_cte_materialization_specifiers if defined?(@_supports_cte_materialization_specifiers)
|
145
172
|
|
146
173
|
@_supports_cte_materialization_specifiers = connection.postgresql_version >= 120000
|
147
174
|
end
|
175
|
+
|
176
|
+
# Postgres advisory unlocking function for the class
|
177
|
+
# @param function [String, Symbol] name of advisory lock or unlock function
|
178
|
+
# @return [Boolean]
|
179
|
+
def advisory_unlockable_function(function = advisory_lockable_function)
|
180
|
+
function.to_s.sub("_lock", "_unlock").sub("_try_", "_")
|
181
|
+
end
|
182
|
+
|
183
|
+
# Unlocks all advisory locks active in the current database session/connection
|
184
|
+
# @return [void]
|
185
|
+
def advisory_unlock_session
|
186
|
+
connection.exec_query("SELECT pg_advisory_unlock_all()::text AS unlocked", 'GoodJob::Lockable Unlock Session').first[:unlocked]
|
187
|
+
end
|
188
|
+
|
189
|
+
# Converts SQL query strings between PG-compatible and JDBC-compatible syntax
|
190
|
+
# @param query [String]
|
191
|
+
# @return [Boolean]
|
192
|
+
def pg_or_jdbc_query(query)
|
193
|
+
if Concurrent.on_jruby?
|
194
|
+
# Replace $1 bind parameters with ?
|
195
|
+
query.gsub(/\$\d*/, '?')
|
196
|
+
else
|
197
|
+
query
|
198
|
+
end
|
199
|
+
end
|
148
200
|
end
|
149
201
|
|
150
202
|
# Acquires an advisory lock on this record if it is not already locked by
|
151
203
|
# another database session. Be careful to ensure you release the lock when
|
152
204
|
# you are done with {#advisory_unlock} (or {#advisory_unlock!} to release
|
153
205
|
# all remaining locks).
|
206
|
+
# @param key [String, Symbol] Key to Advisory Lock against
|
207
|
+
# @param function [String, Symbol] Postgres Advisory Lock function name to use
|
154
208
|
# @return [Boolean] whether the lock was acquired.
|
155
|
-
def advisory_lock
|
156
|
-
query =
|
157
|
-
|
158
|
-
|
159
|
-
|
160
|
-
|
161
|
-
|
209
|
+
def advisory_lock(key: lockable_key, function: advisory_lockable_function)
|
210
|
+
query = if function.include? "_try_"
|
211
|
+
<<~SQL.squish
|
212
|
+
SELECT #{function}(('x'||substr(md5($1::text), 1, 16))::bit(64)::bigint) AS locked
|
213
|
+
SQL
|
214
|
+
else
|
215
|
+
<<~SQL.squish
|
216
|
+
SELECT #{function}(('x'||substr(md5($1::text), 1, 16))::bit(64)::bigint)::text AS locked
|
217
|
+
SQL
|
218
|
+
end
|
219
|
+
|
220
|
+
binds = [[nil, key]]
|
221
|
+
self.class.connection.exec_query(pg_or_jdbc_query(query), 'GoodJob::Lockable Advisory Lock', binds).first['locked']
|
162
222
|
end
|
163
223
|
|
164
224
|
# Releases an advisory lock on this record if it is locked by this database
|
165
225
|
# session. Note that advisory locks stack, so you must call
|
166
226
|
# {#advisory_unlock} and {#advisory_lock} the same number of times.
|
227
|
+
# @param key [String, Symbol] Key to lock against
|
228
|
+
# @param function [String, Symbol] Postgres Advisory Lock function name to use
|
167
229
|
# @return [Boolean] whether the lock was released.
|
168
|
-
def advisory_unlock
|
230
|
+
def advisory_unlock(key: lockable_key, function: self.class.advisory_unlockable_function(advisory_lockable_function))
|
169
231
|
query = <<~SQL.squish
|
170
|
-
SELECT 1 AS
|
171
|
-
WHERE pg_advisory_unlock(('x'||substr(md5($1 || $2::text), 1, 16))::bit(64)::bigint)
|
232
|
+
SELECT #{function}(('x'||substr(md5($1::text), 1, 16))::bit(64)::bigint) AS unlocked
|
172
233
|
SQL
|
173
|
-
binds = [[nil,
|
174
|
-
self.class.connection.exec_query(pg_or_jdbc_query(query), 'GoodJob::Lockable Advisory Unlock', binds).
|
234
|
+
binds = [[nil, key]]
|
235
|
+
self.class.connection.exec_query(pg_or_jdbc_query(query), 'GoodJob::Lockable Advisory Unlock', binds).first['unlocked']
|
175
236
|
end
|
176
237
|
|
177
238
|
# Acquires an advisory lock on this record or raises
|
178
239
|
# {RecordAlreadyAdvisoryLockedError} if it is already locked by another
|
179
240
|
# database session.
|
241
|
+
# @param key [String, Symbol] Key to lock against
|
242
|
+
# @param function [String, Symbol] Postgres Advisory Lock function name to use
|
180
243
|
# @raise [RecordAlreadyAdvisoryLockedError]
|
181
244
|
# @return [Boolean] +true+
|
182
|
-
def advisory_lock!
|
183
|
-
result = advisory_lock
|
245
|
+
def advisory_lock!(key: lockable_key, function: advisory_lockable_function)
|
246
|
+
result = advisory_lock(key: key, function: function)
|
184
247
|
result || raise(RecordAlreadyAdvisoryLockedError)
|
185
248
|
end
|
186
249
|
|
187
250
|
# Acquires an advisory lock on this record and safely releases it after the
|
188
251
|
# passed block is completed. If the record is locked by another database
|
189
252
|
# session, this raises {RecordAlreadyAdvisoryLockedError}.
|
190
|
-
#
|
253
|
+
# @param key [String, Symbol] Key to lock against
|
254
|
+
# @param function [String, Symbol] Postgres Advisory Lock function name to use
|
191
255
|
# @yield Nothing
|
192
256
|
# @return [Object] The result of the block.
|
193
257
|
#
|
@@ -196,64 +260,63 @@ module GoodJob
|
|
196
260
|
# record.with_advisory_lock do
|
197
261
|
# do_something_with record
|
198
262
|
# end
|
199
|
-
def with_advisory_lock
|
263
|
+
def with_advisory_lock(key: lockable_key, function: advisory_lockable_function)
|
200
264
|
raise ArgumentError, "Must provide a block" unless block_given?
|
201
265
|
|
202
|
-
advisory_lock!
|
266
|
+
advisory_lock!(key: key, function: function)
|
203
267
|
yield
|
204
268
|
ensure
|
205
|
-
advisory_unlock unless $ERROR_INFO.is_a? RecordAlreadyAdvisoryLockedError
|
269
|
+
advisory_unlock(key: key, function: self.class.advisory_unlockable_function(function)) unless $ERROR_INFO.is_a? RecordAlreadyAdvisoryLockedError
|
206
270
|
end
|
207
271
|
|
208
272
|
# Tests whether this record has an advisory lock on it.
|
273
|
+
# @param key [String, Symbol] Key to test lock against
|
209
274
|
# @return [Boolean]
|
210
|
-
def advisory_locked?
|
275
|
+
def advisory_locked?(key: lockable_key)
|
211
276
|
query = <<~SQL.squish
|
212
277
|
SELECT 1 AS one
|
213
278
|
FROM pg_locks
|
214
279
|
WHERE pg_locks.locktype = 'advisory'
|
215
280
|
AND pg_locks.objsubid = 1
|
216
|
-
AND pg_locks.classid = ('x' || substr(md5($1
|
217
|
-
AND pg_locks.objid = (('x' || substr(md5($
|
281
|
+
AND pg_locks.classid = ('x' || substr(md5($1::text), 1, 16))::bit(32)::int
|
282
|
+
AND pg_locks.objid = (('x' || substr(md5($2::text), 1, 16))::bit(64) << 32)::bit(32)::int
|
218
283
|
SQL
|
219
|
-
binds = [[nil,
|
284
|
+
binds = [[nil, key], [nil, key]]
|
220
285
|
self.class.connection.exec_query(pg_or_jdbc_query(query), 'GoodJob::Lockable Advisory Locked?', binds).any?
|
221
286
|
end
|
222
287
|
|
223
288
|
# Tests whether this record is locked by the current database session.
|
289
|
+
# @param key [String, Symbol] Key to test lock against
|
224
290
|
# @return [Boolean]
|
225
|
-
def owns_advisory_lock?
|
291
|
+
def owns_advisory_lock?(key: lockable_key)
|
226
292
|
query = <<~SQL.squish
|
227
293
|
SELECT 1 AS one
|
228
294
|
FROM pg_locks
|
229
295
|
WHERE pg_locks.locktype = 'advisory'
|
230
296
|
AND pg_locks.objsubid = 1
|
231
|
-
AND pg_locks.classid = ('x' || substr(md5($1
|
232
|
-
AND pg_locks.objid = (('x' || substr(md5($
|
297
|
+
AND pg_locks.classid = ('x' || substr(md5($1::text), 1, 16))::bit(32)::int
|
298
|
+
AND pg_locks.objid = (('x' || substr(md5($2::text), 1, 16))::bit(64) << 32)::bit(32)::int
|
233
299
|
AND pg_locks.pid = pg_backend_pid()
|
234
300
|
SQL
|
235
|
-
binds = [[nil,
|
301
|
+
binds = [[nil, key], [nil, key]]
|
236
302
|
self.class.connection.exec_query(pg_or_jdbc_query(query), 'GoodJob::Lockable Owns Advisory Lock?', binds).any?
|
237
303
|
end
|
238
304
|
|
239
305
|
# Releases all advisory locks on the record that are held by the current
|
240
306
|
# database session.
|
307
|
+
# @param key [String, Symbol] Key to lock against
|
308
|
+
# @param function [String, Symbol] Postgres Advisory Lock function name to use
|
241
309
|
# @return [void]
|
242
|
-
def advisory_unlock!
|
243
|
-
advisory_unlock while advisory_locked?
|
310
|
+
def advisory_unlock!(key: lockable_key, function: self.class.advisory_unlockable_function(advisory_lockable_function))
|
311
|
+
advisory_unlock(key: key, function: function) while advisory_locked?
|
244
312
|
end
|
245
313
|
|
246
|
-
|
247
|
-
|
248
|
-
|
249
|
-
|
250
|
-
def pg_or_jdbc_query(query)
|
251
|
-
if Concurrent.on_jruby?
|
252
|
-
# Replace $1 bind parameters with ?
|
253
|
-
query.gsub(/\$\d*/, '?')
|
254
|
-
else
|
255
|
-
query
|
256
|
-
end
|
314
|
+
# Default Advisory Lock key
|
315
|
+
# @return [String]
|
316
|
+
def lockable_key
|
317
|
+
[self.class.table_name, self[self.class._advisory_lockable_column]].join
|
257
318
|
end
|
319
|
+
|
320
|
+
delegate :pg_or_jdbc_query, to: :class
|
258
321
|
end
|
259
322
|
end
|
data/lib/good_job/poller.rb
CHANGED
@@ -5,11 +5,13 @@ module GoodJob # :nodoc:
|
|
5
5
|
# Pollers regularly wake up execution threads to check for new work.
|
6
6
|
#
|
7
7
|
class Poller
|
8
|
+
TIMEOUT_INTERVAL = 5
|
9
|
+
|
8
10
|
# Defaults for instance of Concurrent::TimerTask.
|
9
11
|
# The timer controls how and when sleeping threads check for new work.
|
10
12
|
DEFAULT_TIMER_OPTIONS = {
|
11
13
|
execution_interval: Configuration::DEFAULT_POLL_INTERVAL,
|
12
|
-
timeout_interval:
|
14
|
+
timeout_interval: TIMEOUT_INTERVAL,
|
13
15
|
run_now: true,
|
14
16
|
}.freeze
|
15
17
|
|
@@ -49,9 +51,11 @@ module GoodJob # :nodoc:
|
|
49
51
|
|
50
52
|
# Tests whether the timer is shutdown.
|
51
53
|
# @return [true, false, nil]
|
52
|
-
|
54
|
+
def shutdown?
|
55
|
+
timer ? timer.shutdown? : true
|
56
|
+
end
|
53
57
|
|
54
|
-
# Shut down the
|
58
|
+
# Shut down the poller.
|
55
59
|
# Use {#shutdown?} to determine whether threads have stopped.
|
56
60
|
# @param timeout [nil, Numeric] Seconds to wait for active threads.
|
57
61
|
# * +nil+, the scheduler will trigger a shutdown but not wait for it to complete.
|
data/lib/good_job/version.rb
CHANGED
metadata
CHANGED
@@ -1,14 +1,14 @@
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
2
2
|
name: good_job
|
3
3
|
version: !ruby/object:Gem::Version
|
4
|
-
version: 1.
|
4
|
+
version: 1.11.1
|
5
5
|
platform: ruby
|
6
6
|
authors:
|
7
7
|
- Ben Sheldon
|
8
8
|
autorequire:
|
9
9
|
bindir: exe
|
10
10
|
cert_chain: []
|
11
|
-
date: 2021-
|
11
|
+
date: 2021-07-07 00:00:00.000000000 Z
|
12
12
|
dependencies:
|
13
13
|
- !ruby/object:Gem::Dependency
|
14
14
|
name: activejob
|
@@ -94,6 +94,20 @@ dependencies:
|
|
94
94
|
- - ">="
|
95
95
|
- !ruby/object:Gem::Version
|
96
96
|
version: '2.0'
|
97
|
+
- !ruby/object:Gem::Dependency
|
98
|
+
name: benchmark-ips
|
99
|
+
requirement: !ruby/object:Gem::Requirement
|
100
|
+
requirements:
|
101
|
+
- - '='
|
102
|
+
- !ruby/object:Gem::Version
|
103
|
+
version: 2.8.4
|
104
|
+
type: :development
|
105
|
+
prerelease: false
|
106
|
+
version_requirements: !ruby/object:Gem::Requirement
|
107
|
+
requirements:
|
108
|
+
- - '='
|
109
|
+
- !ruby/object:Gem::Version
|
110
|
+
version: 2.8.4
|
97
111
|
- !ruby/object:Gem::Dependency
|
98
112
|
name: capybara
|
99
113
|
requirement: !ruby/object:Gem::Requirement
|
@@ -327,19 +341,29 @@ files:
|
|
327
341
|
- engine/app/controllers/good_job/assets_controller.rb
|
328
342
|
- engine/app/controllers/good_job/base_controller.rb
|
329
343
|
- engine/app/controllers/good_job/dashboards_controller.rb
|
344
|
+
- engine/app/controllers/good_job/jobs_controller.rb
|
330
345
|
- engine/app/helpers/good_job/application_helper.rb
|
331
346
|
- engine/app/views/good_job/active_jobs/show.html.erb
|
332
347
|
- engine/app/views/good_job/dashboards/index.html.erb
|
333
348
|
- engine/app/views/layouts/good_job/base.html.erb
|
334
349
|
- engine/app/views/shared/_chart.erb
|
335
350
|
- engine/app/views/shared/_jobs_table.erb
|
351
|
+
- engine/app/views/shared/icons/_check.html.erb
|
352
|
+
- engine/app/views/shared/icons/_exclamation.html.erb
|
353
|
+
- engine/app/views/shared/icons/_trash.html.erb
|
336
354
|
- engine/config/routes.rb
|
337
355
|
- engine/lib/good_job/engine.rb
|
338
356
|
- exe/good_job
|
339
357
|
- lib/active_job/queue_adapters/good_job_adapter.rb
|
340
358
|
- lib/generators/good_job/install_generator.rb
|
341
|
-
- lib/generators/good_job/templates/
|
359
|
+
- lib/generators/good_job/templates/install/migrations/create_good_jobs.rb.erb
|
360
|
+
- lib/generators/good_job/templates/update/migrations/01_create_good_jobs.rb
|
361
|
+
- lib/generators/good_job/templates/update/migrations/02_add_active_job_id_concurrency_key_cron_key_to_good_jobs.rb
|
362
|
+
- lib/generators/good_job/templates/update/migrations/03_add_active_job_id_index_and_concurrency_key_index_to_good_jobs.rb
|
363
|
+
- lib/generators/good_job/update_generator.rb
|
342
364
|
- lib/good_job.rb
|
365
|
+
- lib/good_job/active_job_extensions.rb
|
366
|
+
- lib/good_job/active_job_extensions/concurrency.rb
|
343
367
|
- lib/good_job/adapter.rb
|
344
368
|
- lib/good_job/cli.rb
|
345
369
|
- lib/good_job/configuration.rb
|