good_job 4.14.2 → 4.16.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/CHANGELOG.md +43 -0
- data/README.md +128 -19
- data/app/controllers/good_job/cleaner_controller.rb +2 -2
- data/app/filters/good_job/base_filter.rb +1 -0
- data/app/filters/good_job/jobs_filter.rb +61 -35
- data/app/models/concerns/good_job/advisory_lockable.rb +356 -123
- data/app/models/concerns/good_job/error_events.rb +13 -6
- data/app/models/concerns/good_job/filterable.rb +2 -2
- data/app/models/good_job/batch.rb +67 -36
- data/app/models/good_job/execution.rb +6 -0
- data/app/models/good_job/execution_result.rb +4 -1
- data/app/models/good_job/job/lockable.rb +118 -0
- data/app/models/good_job/job.rb +163 -32
- data/app/views/good_job/jobs/_executions.erb +2 -0
- data/app/views/good_job/jobs/_table.erb +1 -1
- data/app/views/good_job/shared/_filter.erb +6 -1
- data/config/brakeman.ignore +21 -46
- data/config/locales/de.yml +2 -0
- data/config/locales/en.yml +2 -0
- data/config/locales/es.yml +2 -0
- data/config/locales/fr.yml +2 -0
- data/config/locales/it.yml +2 -0
- data/config/locales/ja.yml +2 -0
- data/config/locales/ko.yml +2 -0
- data/config/locales/nl.yml +2 -0
- data/config/locales/pt-BR.yml +2 -0
- data/config/locales/ru.yml +2 -0
- data/config/locales/tr.yml +2 -0
- data/config/locales/uk.yml +2 -0
- data/config/locales/zh-CN.yml +2 -0
- data/lib/generators/good_job/templates/install/migrations/create_good_jobs.rb.erb +10 -0
- data/lib/generators/good_job/templates/update/migrations/01_create_good_jobs.rb.erb +5 -0
- data/lib/generators/good_job/templates/update/migrations/07_add_lock_type_to_good_jobs.rb.erb +7 -0
- data/lib/generators/good_job/templates/update/migrations/08_add_index_good_jobs_for_candidate_dequeue_unlocked.rb.erb +14 -0
- data/lib/generators/good_job/templates/update/migrations/09_add_index_good_jobs_priority_scheduled_at.rb.erb +23 -0
- data/lib/good_job/active_job_extensions/concurrency.rb +181 -89
- data/lib/good_job/adapter.rb +81 -33
- data/lib/good_job/configuration.rb +24 -0
- data/lib/good_job/current_thread.rb +7 -0
- data/lib/good_job/engine.rb +4 -0
- data/lib/good_job/job_performer.rb +1 -1
- data/lib/good_job/notifier.rb +1 -0
- data/lib/good_job/version.rb +1 -1
- data/lib/good_job.rb +6 -2
- metadata +6 -2
checksums.yaml
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
SHA256:
|
|
3
|
-
metadata.gz:
|
|
4
|
-
data.tar.gz:
|
|
3
|
+
metadata.gz: 9b13d49cf0a8c78543c55426eaf82eafd8518bd6f9a22ab081ac1657ff129227
|
|
4
|
+
data.tar.gz: c8e21ac65881fa645409b00d496beff08b9f99cfbcefb832a01d29bcff2e760a
|
|
5
5
|
SHA512:
|
|
6
|
-
metadata.gz:
|
|
7
|
-
data.tar.gz:
|
|
6
|
+
metadata.gz: af467c3effc78f424155420005e79ffdbff6249ae7a02ef9084f94d81d8aa28920bcfbc71f46349f6e67597f73cf39f7f2f2cd7f4edd869084091e9ab962c951
|
|
7
|
+
data.tar.gz: 19732a4dbfa6c64aeddd85c721a82f94cbaf50f2e784fa04bf22d2321bb518723992535595cdcc32a172cefbb77476ea66c9e06182c640dc2dc62b0306bf2cae
|
data/CHANGELOG.md
CHANGED
|
@@ -1,5 +1,48 @@
|
|
|
1
1
|
# Changelog
|
|
2
2
|
|
|
3
|
+
## [v4.16.0](https://github.com/bensheldon/good_job/tree/v4.16.0) (2026-04-14)
|
|
4
|
+
|
|
5
|
+
[Full Changelog](https://github.com/bensheldon/good_job/compare/v4.15.0...v4.16.0)
|
|
6
|
+
|
|
7
|
+
**Implemented enhancements:**
|
|
8
|
+
|
|
9
|
+
- Allow filtering by label on dashboard [\#1739](https://github.com/bensheldon/good_job/pull/1739) ([bensheldon](https://github.com/bensheldon))
|
|
10
|
+
- Allow multiple concurrency rules per job via labels [\#1700](https://github.com/bensheldon/good_job/pull/1700) ([bscofield](https://github.com/bscofield))
|
|
11
|
+
|
|
12
|
+
**Fixed bugs:**
|
|
13
|
+
|
|
14
|
+
- Fix advisory lock connection stickiness in block contexts [\#1736](https://github.com/bensheldon/good_job/pull/1736) ([bensheldon](https://github.com/bensheldon))
|
|
15
|
+
- Add JRuby 10 to testing matrix [\#1559](https://github.com/bensheldon/good_job/pull/1559) ([bensheldon](https://github.com/bensheldon))
|
|
16
|
+
|
|
17
|
+
**Closed issues:**
|
|
18
|
+
|
|
19
|
+
- Job duration misreported if interrupted [\#1723](https://github.com/bensheldon/good_job/issues/1723)
|
|
20
|
+
|
|
21
|
+
**Merged pull requests:**
|
|
22
|
+
|
|
23
|
+
- Use annotated git tag in release script [\#1741](https://github.com/bensheldon/good_job/pull/1741) ([bensheldon](https://github.com/bensheldon))
|
|
24
|
+
- Double single-thread scheduler integration test timeout on JRuby [\#1738](https://github.com/bensheldon/good_job/pull/1738) ([bensheldon](https://github.com/bensheldon))
|
|
25
|
+
- Fix JRuby test flakes for scheduler timeout and interrupted execution duration [\#1737](https://github.com/bensheldon/good_job/pull/1737) ([bensheldon](https://github.com/bensheldon))
|
|
26
|
+
- Count Advisory Locks and refactor advisory lock lifecycle [\#1735](https://github.com/bensheldon/good_job/pull/1735) ([bensheldon](https://github.com/bensheldon))
|
|
27
|
+
- Show interrupted execution recovery duration in dashboard [\#1733](https://github.com/bensheldon/good_job/pull/1733) ([bensheldon](https://github.com/bensheldon))
|
|
28
|
+
- chore: use `merge` to avoid mutate the query object [\#1717](https://github.com/bensheldon/good_job/pull/1717) ([luizkowalski](https://github.com/luizkowalski))
|
|
29
|
+
|
|
30
|
+
## [v4.15.0](https://github.com/bensheldon/good_job/tree/v4.15.0) (2026-04-09)
|
|
31
|
+
|
|
32
|
+
[Full Changelog](https://github.com/bensheldon/good_job/compare/v4.14.2...v4.15.0)
|
|
33
|
+
|
|
34
|
+
**Implemented enhancements:**
|
|
35
|
+
|
|
36
|
+
- Add opt-in "FOR NO KEY UPDATE SKIP LOCKED" job lock strategy and hybrid strategy for online migration [\#1731](https://github.com/bensheldon/good_job/pull/1731) ([bensheldon](https://github.com/bensheldon))
|
|
37
|
+
- Allow ordering by scheduled\_at instead of created\_at when dequeueing job [\#1645](https://github.com/bensheldon/good_job/pull/1645) ([lsylvester](https://github.com/lsylvester))
|
|
38
|
+
- Allow `GoodJob.preserve_job_records` to take a lambda that is callable after each job executes [\#1640](https://github.com/bensheldon/good_job/pull/1640) ([bensheldon](https://github.com/bensheldon))
|
|
39
|
+
|
|
40
|
+
**Merged pull requests:**
|
|
41
|
+
|
|
42
|
+
- Fix JRuby in development lockfile, with test [\#1734](https://github.com/bensheldon/good_job/pull/1734) ([bensheldon](https://github.com/bensheldon))
|
|
43
|
+
- Add herb to linter [\#1732](https://github.com/bensheldon/good_job/pull/1732) ([bensheldon](https://github.com/bensheldon))
|
|
44
|
+
- Update development dependencies; apply Rubocop to\_h lints [\#1728](https://github.com/bensheldon/good_job/pull/1728) ([bensheldon](https://github.com/bensheldon))
|
|
45
|
+
|
|
3
46
|
## [v4.14.2](https://github.com/bensheldon/good_job/tree/v4.14.2) (2026-04-06)
|
|
4
47
|
|
|
5
48
|
[Full Changelog](https://github.com/bensheldon/good_job/compare/v4.14.1...v4.14.2)
|
data/README.md
CHANGED
|
@@ -303,13 +303,13 @@ Available configuration options are:
|
|
|
303
303
|
- `cron_graceful_restart_period` (integer) when restarting cron, attempt to re-enqueue jobs that would have been enqueued by cron within this time period (e.g. `1.minute`). This should match the expected downtime during deploys.
|
|
304
304
|
- `enable_listen_notify` (boolean) whether to enqueue and read jobs with Postgres LISTEN/NOTIFY. Defaults to `true`. You can also set this with the environment variable `GOOD_JOB_ENABLE_LISTEN_NOTIFY`.
|
|
305
305
|
- `cron` (hash) cron configuration. Defaults to `{}`. You can also set this as a JSON string with the environment variable `GOOD_JOB_CRON`
|
|
306
|
-
- `cleanup_discarded_jobs` (boolean) whether to destroy discarded jobs when cleaning up preserved jobs using the `$ good_job cleanup_preserved_jobs` CLI command or calling `GoodJob.cleanup_preserved_jobs`. Defaults to `true`. Can also be set with
|
|
307
|
-
- `cleanup_preserved_jobs_before_seconds_ago` (integer) number of seconds to preserve jobs when using the `$ good_job cleanup_preserved_jobs` CLI command or calling `GoodJob.cleanup_preserved_jobs`. Defaults to `1209600` (14 days). Can also be set with the environment variable `GOOD_JOB_CLEANUP_PRESERVED_JOBS_BEFORE_SECONDS_AGO`.
|
|
306
|
+
- `cleanup_discarded_jobs` (boolean) whether to destroy discarded jobs when cleaning up preserved jobs using the `$ good_job cleanup_preserved_jobs` CLI command or calling `GoodJob.cleanup_preserved_jobs`. Defaults to `true`. Can also be set with the environment variable `GOOD_JOB_CLEANUP_DISCARDED_JOBS`.
|
|
307
|
+
- `cleanup_preserved_jobs_before_seconds_ago` (integer) number of seconds to preserve jobs when using the `$ good_job cleanup_preserved_jobs` CLI command or calling `GoodJob.cleanup_preserved_jobs`. Defaults to `1209600` (14 days). Can also be set with the environment variable `GOOD_JOB_CLEANUP_PRESERVED_JOBS_BEFORE_SECONDS_AGO`.
|
|
308
308
|
- `cleanup_interval_jobs` (integer) Number of jobs a Scheduler will execute before cleaning up preserved jobs. Defaults to `1000`. Disable with `false`. Can also be set with the environment variable `GOOD_JOB_CLEANUP_INTERVAL_JOBS` and disabled with `0`).
|
|
309
309
|
- `cleanup_interval_seconds` (integer) Number of seconds a Scheduler will wait before cleaning up preserved jobs. Defaults to `600` (10 minutes). Disable with `false`. Can also be set with the environment variable `GOOD_JOB_CLEANUP_INTERVAL_SECONDS` and disabled with `0`).
|
|
310
310
|
- `inline_execution_respects_schedule` (boolean) Opt-in to future behavior of inline execution respecting scheduled jobs. Defaults to `false`.
|
|
311
311
|
- `logger` ([Rails Logger](https://api.rubyonrails.org/classes/ActiveSupport/Logger.html)) lets you set a custom logger for GoodJob. It should be an instance of a Rails `Logger` (Default: `Rails.logger`).
|
|
312
|
-
- `preserve_job_records` (boolean) keeps job records in your database even after jobs are completed. (Default: `true`)
|
|
312
|
+
- `preserve_job_records` (boolean, symbol, or lambda) keeps job records in your database even after jobs are completed. If set to `true`, all job records are preserved. If set to `:on_unhandled_error`, only jobs that finished with an unhandled error are preserved. If set to a lambda, the lambda will be called with the error_event (e.g., `:discarded`, `:retry_stopped`, or `:unhandled`) and should return a boolean indicating whether to preserve the job. (Default: `true`)
|
|
313
313
|
- `advisory_lock_heartbeat` (boolean) whether to use an advisory lock for the purpose of determining whether an execeution process is active. (Default `true` in Development; `false` in other environments)
|
|
314
314
|
- `retry_on_unhandled_error` (boolean) causes jobs to be re-queued and retried if they raise an instance of `StandardError`. Be advised this may lead to jobs being repeated infinitely ([see below for more on retries](#retries)). Instances of `Exception`, like SIGINT, will *always* be retried, regardless of this attribute’s value. (Default: `false`)
|
|
315
315
|
- `on_thread_error` (proc, lambda, or callable) will be called when there is an Exception. It can be useful for logging errors to bug tracking services, like Sentry or Airbrake. Example:
|
|
@@ -375,7 +375,7 @@ GoodJob.active_record_parent_class = "ApplicationRecord"
|
|
|
375
375
|
The following options are also configurable via accessors, but you are encouraged to use the configuration attributes instead because these may be deprecated and removed in the future:
|
|
376
376
|
|
|
377
377
|
- **`GoodJob.logger`** ([Rails Logger](https://api.rubyonrails.org/classes/ActiveSupport/Logger.html)) lets you set a custom logger for GoodJob. It should be an instance of a Rails `Logger`.
|
|
378
|
-
- **`GoodJob.preserve_job_records`** (boolean) keeps job records in your database even after jobs are completed. (Default: `true`)
|
|
378
|
+
- **`GoodJob.preserve_job_records`** (boolean, symbol, or lambda) keeps job records in your database even after jobs are completed. If set to `true`, all job records are preserved. If set to `:on_unhandled_error`, only jobs that finished with an unhandled error are preserved. If set to a lambda, the lambda will be called with Active Job instance, and if it exists, the exception the error_event (e.g., `:discarded`, `:retry_stopped`, or `:unhandled`) and should return a boolean indicating whether to preserve the job (e.g. `-> (active_job, error, error_event) { !(active_job.is_a(Turbo::Streams::BroadcastStreamJob) || error_event == :retry_stopped`). (Default: `true`)
|
|
379
379
|
- **`GoodJob.retry_on_unhandled_error`** (boolean) causes jobs to be re-queued and retried if they raise an instance of `StandardError`. Be advised this may lead to jobs being repeated infinitely ([see below for more on retries](#retries)). Instances of `Exception`, like SIGINT, will *always* be retried, regardless of this attribute’s value. (Default: `false`)
|
|
380
380
|
- **`GoodJob.on_thread_error`** (proc, lambda, or callable) will be called when there is an Exception. It can be useful for logging errors to bug tracking services, like Sentry or Airbrake.
|
|
381
381
|
|
|
@@ -538,6 +538,78 @@ Labels can be used to search jobs in the Dashboard. For example, to find all job
|
|
|
538
538
|
|
|
539
539
|
GoodJob can extend Active Job to provide limits on concurrently running jobs, either at time of _enqueue_ or at _perform_. Limiting concurrency can help prevent duplicate, double or unnecessary jobs from being enqueued, or race conditions when performing, for example when interacting with 3rd-party APIs.
|
|
540
540
|
|
|
541
|
+
```ruby
|
|
542
|
+
class MyJob < ApplicationJob
|
|
543
|
+
include GoodJob::ActiveJobExtensions::Concurrency
|
|
544
|
+
|
|
545
|
+
# Define one or more concurrency rules. Each rule is scoped to a label,
|
|
546
|
+
# which is a value derived from the job's arguments at enqueue time and
|
|
547
|
+
# stored on the job record. Jobs must be enqueued with the matching label
|
|
548
|
+
# via `good_job_labels:` for the rule to apply.
|
|
549
|
+
#
|
|
550
|
+
# Multiple rules can be defined; they are evaluated in order and the first
|
|
551
|
+
# exceeded rule short-circuits the rest.
|
|
552
|
+
good_job_concurrency_rule(
|
|
553
|
+
# A label that scopes this rule. Can be a static String or a Lambda/Proc
|
|
554
|
+
# invoked in the context of the job instance. The rule only applies to jobs
|
|
555
|
+
# that were enqueued with this label in `good_job_labels`.
|
|
556
|
+
label: -> { arguments.first[:user_id] },
|
|
557
|
+
|
|
558
|
+
# Maximum number of unfinished jobs with this label to allow.
|
|
559
|
+
# Can be an Integer or Lambda/Proc invoked in the context of the job.
|
|
560
|
+
total_limit: 1,
|
|
561
|
+
|
|
562
|
+
# Or, if more control is needed:
|
|
563
|
+
# Maximum number of jobs with this label to be concurrently enqueued
|
|
564
|
+
# (excludes performing jobs). Can be an Integer or Lambda/Proc.
|
|
565
|
+
enqueue_limit: 2,
|
|
566
|
+
|
|
567
|
+
# Maximum number of jobs with this label to be concurrently performed
|
|
568
|
+
# (excludes enqueued jobs). Can be an Integer or Lambda/Proc.
|
|
569
|
+
perform_limit: 1,
|
|
570
|
+
|
|
571
|
+
# Maximum number of jobs with this label to be enqueued within the time
|
|
572
|
+
# period, looking backwards from now. Must be [count, period].
|
|
573
|
+
enqueue_throttle: [10, 1.minute],
|
|
574
|
+
|
|
575
|
+
# Maximum number of jobs with this label to be performed within the time
|
|
576
|
+
# period, looking backwards from now. Must be [count, period].
|
|
577
|
+
perform_throttle: [100, 1.hour],
|
|
578
|
+
|
|
579
|
+
# Note: Under heavy load, the total number of jobs may exceed the
|
|
580
|
+
# sum of `enqueue_limit` and `perform_limit` because of race conditions
|
|
581
|
+
# caused by imperfectly disjunctive states. If you need to constrain
|
|
582
|
+
# the total number of jobs, use `total_limit` instead. See #378.
|
|
583
|
+
)
|
|
584
|
+
# Additional rules
|
|
585
|
+
good_job_concurrency_rule(...)
|
|
586
|
+
good_job_concurrency_rule(...)
|
|
587
|
+
|
|
588
|
+
def perform(user_id:)
|
|
589
|
+
# do work
|
|
590
|
+
end
|
|
591
|
+
end
|
|
592
|
+
```
|
|
593
|
+
|
|
594
|
+
Jobs must be enqueued with the matching label for rules to take effect:
|
|
595
|
+
|
|
596
|
+
```ruby
|
|
597
|
+
MyJob.set(good_job_labels: [current_user.id]).perform_later(user_id: current_user.id)
|
|
598
|
+
```
|
|
599
|
+
|
|
600
|
+
#### How concurrency controls work
|
|
601
|
+
|
|
602
|
+
GoodJob's concurrency control strategy for `perform_limit` is "optimistic retry with an incremental backoff". The [code is readable](https://github.com/bensheldon/good_job/blob/main/lib/good_job/active_job_extensions/concurrency.rb).
|
|
603
|
+
|
|
604
|
+
- "Optimistic" meaning that the implementation's performance trade-off assumes that collisions are atypical (e.g. two users enqueue the same job at the same time) rather than regular (e.g. the system enqueues thousands of colliding jobs at the same time). Depending on your concurrency requirements, you may also want to manage concurrency through the number of GoodJob threads and processes that are performing a given queue.
|
|
605
|
+
- "Retry with an incremental backoff" means that when `perform_limit` is exceeded, the job will raise a `GoodJob::ActiveJobExtensions::Concurrency::ConcurrencyExceededError` which is caught by a `retry_on` handler which re-schedules the job to execute in the near future with an incremental backoff.
|
|
606
|
+
- First-in-first-out job execution order is not preserved when a job is retried with incremental back-off.
|
|
607
|
+
- For pessimistic usecases that collisions are expected, use number of threads/processes (e.g., `good_job --queues "serial:1;-serial:5"`) to control concurrency. It is also a good idea to use `perform_limit` as backstop.
|
|
608
|
+
|
|
609
|
+
#### Legacy: `good_job_control_concurrency_with`
|
|
610
|
+
|
|
611
|
+
The original concurrency interface uses a single configuration hash and scopes limits to a concurrency _key_ (a string derived from the job) stored on the job record, rather than a label. It remains fully supported.
|
|
612
|
+
|
|
541
613
|
```ruby
|
|
542
614
|
class MyJob < ApplicationJob
|
|
543
615
|
include GoodJob::ActiveJobExtensions::Concurrency
|
|
@@ -568,11 +640,6 @@ class MyJob < ApplicationJob
|
|
|
568
640
|
# with two elements: the number of jobs and the time period.
|
|
569
641
|
perform_throttle: [100, 1.hour],
|
|
570
642
|
|
|
571
|
-
# Note: Under heavy load, the total number of jobs may exceed the
|
|
572
|
-
# sum of `enqueue_limit` and `perform_limit` because of race conditions
|
|
573
|
-
# caused by imperfectly disjunctive states. If you need to constrain
|
|
574
|
-
# the total number of jobs, use `total_limit` instead. See #378.
|
|
575
|
-
|
|
576
643
|
# A unique key to be globally locked against.
|
|
577
644
|
# Can be String or Lambda/Proc that is invoked in the context of the job.
|
|
578
645
|
#
|
|
@@ -606,15 +673,6 @@ job = MyJob.perform_later("Alice", version: 'v1')
|
|
|
606
673
|
job.good_job_concurrency_key #=> "MyJob-default-Alice-v1"
|
|
607
674
|
```
|
|
608
675
|
|
|
609
|
-
#### How concurrency controls work
|
|
610
|
-
|
|
611
|
-
GoodJob's concurrency control strategy for `perform_limit` is "optimistic retry with an incremental backoff". The [code is readable](https://github.com/bensheldon/good_job/blob/main/lib/good_job/active_job_extensions/concurrency.rb).
|
|
612
|
-
|
|
613
|
-
- "Optimistic" meaning that the implementation's performance trade-off assumes that collisions are atypical (e.g. two users enqueue the same job at the same time) rather than regular (e.g. the system enqueues thousands of colliding jobs at the same time). Depending on your concurrency requirements, you may also want to manage concurrency through the number of GoodJob threads and processes that are performing a given queue.
|
|
614
|
-
- "Retry with an incremental backoff" means that when `perform_limit` is exceeded, the job will raise a `GoodJob::ActiveJobExtensions::Concurrency::ConcurrencyExceededError` which is caught by a `retry_on` handler which re-schedules the job to execute in the near future with an incremental backoff.
|
|
615
|
-
- First-in-first-out job execution order is not preserved when a job is retried with incremental back-off.
|
|
616
|
-
- For pessimistic usecases that collisions are expected, use number of threads/processes (e.g., `good_job --queues "serial:1;-serial:5"`) to control concurrency. It is also a good idea to use `perform_limit` as backstop.
|
|
617
|
-
|
|
618
676
|
### Cron-style repeating/recurring jobs
|
|
619
677
|
|
|
620
678
|
GoodJob can enqueue Active Job jobs on a recurring basis that can be used as a replacement for cron.
|
|
@@ -1390,7 +1448,11 @@ To instead delete job records immediately after they are finished:
|
|
|
1390
1448
|
|
|
1391
1449
|
```ruby
|
|
1392
1450
|
# config/initializers/good_job.rb
|
|
1393
|
-
config.good_job.preserve_job_records = false # defaults to true; can also be `false
|
|
1451
|
+
config.good_job.preserve_job_records = false # defaults to true; can also be `false`, `:on_unhandled_error`, or a lambda that takes error_event argument
|
|
1452
|
+
|
|
1453
|
+
# Example of using a lambda to preserve only discarded jobs
|
|
1454
|
+
config.good_job.preserve_job_records = ->(error_event) { error_event == :discarded }
|
|
1455
|
+
|
|
1394
1456
|
```
|
|
1395
1457
|
|
|
1396
1458
|
GoodJob will automatically delete preserved job records after 14 days. The retention period, as well as the frequency GoodJob checks for deletable records can be configured:
|
|
@@ -1433,6 +1495,53 @@ travel_to(15.minutes.from_now) { GoodJob.perform_inline }
|
|
|
1433
1495
|
|
|
1434
1496
|
_Note: Rails `travel`/`travel_to` time helpers do not have millisecond precision, so you must leave at least 1 second between the schedule and time traveling for the job to be executed. This [behavior may change in Rails 7.1](https://github.com/rails/rails/pull/44088)._
|
|
1435
1497
|
|
|
1498
|
+
### SKIP LOCKED experimental mode
|
|
1499
|
+
|
|
1500
|
+
By default, GoodJob claims jobs using PostgreSQL advisory locks. As an alternative, GoodJob can use `SELECT FOR UPDATE SKIP LOCKED` to claim jobs, which writes the lock state directly to the `good_jobs` table rather than relying on session-level advisory locks.
|
|
1501
|
+
|
|
1502
|
+
Two strategies are available:
|
|
1503
|
+
|
|
1504
|
+
- **`:skiplocked`** — Claims jobs using `SELECT FOR UPDATE SKIP LOCKED` only. No advisory locks are held. Compatible with PgBouncer in transaction mode.
|
|
1505
|
+
- **`:hybrid`** — Claims jobs using `SELECT FOR UPDATE SKIP LOCKED` and _also_ acquires a session-level advisory lock on the job. Intended for rolling deploys where some workers are still using the default `:advisory` strategy.
|
|
1506
|
+
|
|
1507
|
+
Configure the lock strategy in an initializer or via environment variable:
|
|
1508
|
+
|
|
1509
|
+
```ruby
|
|
1510
|
+
# config/initializers/good_job.rb
|
|
1511
|
+
GoodJob.configure do |config|
|
|
1512
|
+
config.lock_strategy = :skiplocked
|
|
1513
|
+
end
|
|
1514
|
+
```
|
|
1515
|
+
|
|
1516
|
+
```bash
|
|
1517
|
+
GOOD_JOB_LOCK_STRATEGY=skiplocked
|
|
1518
|
+
```
|
|
1519
|
+
|
|
1520
|
+
All three strategies (`:advisory`, `:skiplocked`, `:hybrid`) can coexist safely during a rolling deploy — each strategy excludes jobs that are already locked by another worker regardless of which strategy that worker uses.
|
|
1521
|
+
|
|
1522
|
+
#### PgBouncer configuration
|
|
1523
|
+
|
|
1524
|
+
GoodJob's `:skiplocked` mode makes it compatible with PgBouncer in _transaction_ mode. In addition to setting the lock strategy, you must also disable the `LISTEN/NOTIFY` notifier (which requires a persistent connection) and rely on polling instead:
|
|
1525
|
+
|
|
1526
|
+
```ruby
|
|
1527
|
+
# config/initializers/good_job.rb
|
|
1528
|
+
GoodJob.configure do |config|
|
|
1529
|
+
config.lock_strategy = :skiplocked
|
|
1530
|
+
config.enable_listen_notify = false
|
|
1531
|
+
config.advisory_lock_heartbeat = false
|
|
1532
|
+
config.poll_interval = 5 # seconds; tune based on your latency tolerance
|
|
1533
|
+
end
|
|
1534
|
+
```
|
|
1535
|
+
|
|
1536
|
+
```bash
|
|
1537
|
+
GOOD_JOB_LOCK_STRATEGY=skiplocked
|
|
1538
|
+
GOOD_JOB_ENABLE_LISTEN_NOTIFY=false
|
|
1539
|
+
GOOD_JOB_ADVISORY_LOCK_HEARTBEAT=false
|
|
1540
|
+
GOOD_JOB_POLL_INTERVAL=5
|
|
1541
|
+
```
|
|
1542
|
+
|
|
1543
|
+
With these four settings, GoodJob will not hold any session-level state between queries and is safe to use behind PgBouncer in transaction mode.
|
|
1544
|
+
|
|
1436
1545
|
### PgBouncer compatibility
|
|
1437
1546
|
|
|
1438
1547
|
GoodJob is not compatible with PgBouncer in _transaction_ mode, but is compatible with PgBouncer's _connection_ mode. GoodJob uses connection-based advisory locks and LISTEN/NOTIFY, both of which require full database connections.
|
|
@@ -7,7 +7,7 @@ module GoodJob
|
|
|
7
7
|
|
|
8
8
|
@discarded_jobs_grouped_by_exception =
|
|
9
9
|
GoodJob::Job.discarded
|
|
10
|
-
.select(
|
|
10
|
+
.select(<<~SQL.squish)
|
|
11
11
|
SPLIT_PART(error, ': ', 1) AS exception_class,
|
|
12
12
|
count(id) AS failed,
|
|
13
13
|
COUNT(id) FILTER (WHERE "finished_at" > NOW() - INTERVAL '1 HOUR') AS last_1_hour,
|
|
@@ -21,7 +21,7 @@ module GoodJob
|
|
|
21
21
|
|
|
22
22
|
@discarded_jobs_grouped_by_class =
|
|
23
23
|
GoodJob::Job.discarded
|
|
24
|
-
.select(
|
|
24
|
+
.select(<<~SQL.squish)
|
|
25
25
|
job_class,
|
|
26
26
|
count(id) AS failed,
|
|
27
27
|
COUNT(*) FILTER (WHERE "finished_at" > NOW() - INTERVAL '1 HOUR') AS last_1_hour,
|
|
@@ -23,41 +23,14 @@ module GoodJob
|
|
|
23
23
|
end
|
|
24
24
|
|
|
25
25
|
def filtered_query(filter_params = params)
|
|
26
|
-
|
|
27
|
-
|
|
28
|
-
|
|
29
|
-
|
|
30
|
-
|
|
31
|
-
|
|
32
|
-
|
|
33
|
-
|
|
34
|
-
query.where(active_job_id: search_query)
|
|
35
|
-
else
|
|
36
|
-
query.search_text(search_query)
|
|
37
|
-
end
|
|
38
|
-
end
|
|
39
|
-
|
|
40
|
-
query = query.where(cron_key: filter_params[:cron_key]) if filter_params[:cron_key].present?
|
|
41
|
-
query = query.where(finished_at: finished_since(filter_params[:finished_since])..) if filter_params[:finished_since].present?
|
|
42
|
-
|
|
43
|
-
if filter_params[:state]
|
|
44
|
-
case filter_params[:state]
|
|
45
|
-
when 'discarded'
|
|
46
|
-
query = query.discarded
|
|
47
|
-
when 'succeeded'
|
|
48
|
-
query = query.succeeded
|
|
49
|
-
when 'retried'
|
|
50
|
-
query = query.retried
|
|
51
|
-
when 'scheduled'
|
|
52
|
-
query = query.scheduled
|
|
53
|
-
when 'running'
|
|
54
|
-
query = query.running
|
|
55
|
-
when 'queued'
|
|
56
|
-
query = query.queued
|
|
57
|
-
end
|
|
58
|
-
end
|
|
59
|
-
|
|
60
|
-
query
|
|
26
|
+
base_query
|
|
27
|
+
.merge(filter_by_job_class(filter_params[:job_class]))
|
|
28
|
+
.merge(filter_by_queue_name(filter_params[:queue_name]))
|
|
29
|
+
.merge(filter_by_label(filter_params[:label]))
|
|
30
|
+
.merge(filter_by_search_query(filter_params[:query]))
|
|
31
|
+
.merge(filter_by_cron_key(filter_params[:cron_key]))
|
|
32
|
+
.merge(filter_by_finished_since(filter_params[:finished_since]))
|
|
33
|
+
.merge(filter_by_state(filter_params[:state]))
|
|
61
34
|
end
|
|
62
35
|
|
|
63
36
|
def filtered_count
|
|
@@ -79,6 +52,59 @@ module GoodJob
|
|
|
79
52
|
|
|
80
53
|
private
|
|
81
54
|
|
|
55
|
+
def filter_by_job_class(job_class)
|
|
56
|
+
return {} if job_class.blank?
|
|
57
|
+
|
|
58
|
+
GoodJob::Job.job_class(job_class)
|
|
59
|
+
end
|
|
60
|
+
|
|
61
|
+
def filter_by_queue_name(queue_name)
|
|
62
|
+
return {} if queue_name.blank?
|
|
63
|
+
|
|
64
|
+
GoodJob::Job.where(queue_name: queue_name)
|
|
65
|
+
end
|
|
66
|
+
|
|
67
|
+
def filter_by_label(label)
|
|
68
|
+
return {} if label.blank?
|
|
69
|
+
|
|
70
|
+
GoodJob::Job.labeled(label)
|
|
71
|
+
end
|
|
72
|
+
|
|
73
|
+
def filter_by_search_query(query)
|
|
74
|
+
search_query = query&.strip
|
|
75
|
+
return {} if search_query.blank?
|
|
76
|
+
|
|
77
|
+
if query_is_uuid_and_job_exists?(search_query)
|
|
78
|
+
GoodJob::Job.where(active_job_id: search_query)
|
|
79
|
+
else
|
|
80
|
+
GoodJob::Job.search_text(search_query)
|
|
81
|
+
end
|
|
82
|
+
end
|
|
83
|
+
|
|
84
|
+
def filter_by_cron_key(cron_key)
|
|
85
|
+
return {} if cron_key.blank?
|
|
86
|
+
|
|
87
|
+
GoodJob::Job.where(cron_key: cron_key)
|
|
88
|
+
end
|
|
89
|
+
|
|
90
|
+
def filter_by_finished_since(since)
|
|
91
|
+
return {} if since.blank?
|
|
92
|
+
|
|
93
|
+
GoodJob::Job.where(finished_at: finished_since(since)..)
|
|
94
|
+
end
|
|
95
|
+
|
|
96
|
+
def filter_by_state(state)
|
|
97
|
+
case state
|
|
98
|
+
when 'discarded' then GoodJob::Job.discarded
|
|
99
|
+
when 'succeeded' then GoodJob::Job.succeeded
|
|
100
|
+
when 'retried' then GoodJob::Job.retried
|
|
101
|
+
when 'scheduled' then GoodJob::Job.scheduled
|
|
102
|
+
when 'running' then GoodJob::Job.running
|
|
103
|
+
when 'queued' then GoodJob::Job.queued
|
|
104
|
+
else {}
|
|
105
|
+
end
|
|
106
|
+
end
|
|
107
|
+
|
|
82
108
|
def query_for_records
|
|
83
109
|
filtered_query
|
|
84
110
|
end
|