good_job 4.14.2 → 4.15.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/CHANGELOG.md +16 -0
- data/README.md +56 -5
- data/app/controllers/good_job/cleaner_controller.rb +2 -2
- data/app/models/concerns/good_job/error_events.rb +13 -6
- data/app/models/concerns/good_job/lockable.rb +75 -0
- data/app/models/good_job/batch.rb +67 -36
- data/app/models/good_job/execution_result.rb +4 -1
- data/app/models/good_job/job.rb +178 -32
- data/config/brakeman.ignore +42 -46
- data/lib/generators/good_job/templates/install/migrations/create_good_jobs.rb.erb +10 -0
- data/lib/generators/good_job/templates/update/migrations/01_create_good_jobs.rb.erb +5 -0
- data/lib/generators/good_job/templates/update/migrations/07_add_lock_type_to_good_jobs.rb.erb +7 -0
- data/lib/generators/good_job/templates/update/migrations/08_add_index_good_jobs_for_candidate_dequeue_unlocked.rb.erb +14 -0
- data/lib/generators/good_job/templates/update/migrations/09_add_index_good_jobs_priority_scheduled_at.rb.erb +23 -0
- data/lib/good_job/active_job_extensions/concurrency.rb +3 -3
- data/lib/good_job/adapter.rb +81 -33
- data/lib/good_job/configuration.rb +24 -0
- data/lib/good_job/current_thread.rb +7 -0
- data/lib/good_job/engine.rb +4 -0
- data/lib/good_job/job_performer.rb +1 -1
- data/lib/good_job/version.rb +1 -1
- data/lib/good_job.rb +6 -2
- metadata +6 -2
checksums.yaml
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
SHA256:
|
|
3
|
-
metadata.gz:
|
|
4
|
-
data.tar.gz:
|
|
3
|
+
metadata.gz: d07cdaddabbb6f1966833ae6f79df8a61aef3959c3209ad64ed80e8896d7d56c
|
|
4
|
+
data.tar.gz: 555fdb53f3ba29885d46ac62bc2ab59a0f9cd13186a729c188e13101e7567634
|
|
5
5
|
SHA512:
|
|
6
|
-
metadata.gz:
|
|
7
|
-
data.tar.gz:
|
|
6
|
+
metadata.gz: f4a64d532756884282a5b5ff76b5fb4bc31ceb346f104cab837ef9c946f9a8b6373ae47bfff22d4cdf7785022711b4a2c9a632eb4f8a877fd19dfd674b62a495
|
|
7
|
+
data.tar.gz: 1310d23b3386534c4c5c11f14b2bc521e877a3856371a7b7fb8700f7d03b3a23ecfcc7fa63d4e667628057e2ac6d2c9364f8688788cee8e7538a4ac7216ef1d0
|
data/CHANGELOG.md
CHANGED
|
@@ -1,5 +1,21 @@
|
|
|
1
1
|
# Changelog
|
|
2
2
|
|
|
3
|
+
## [v4.15.0](https://github.com/bensheldon/good_job/tree/v4.15.0) (2026-04-09)
|
|
4
|
+
|
|
5
|
+
[Full Changelog](https://github.com/bensheldon/good_job/compare/v4.14.2...v4.15.0)
|
|
6
|
+
|
|
7
|
+
**Implemented enhancements:**
|
|
8
|
+
|
|
9
|
+
- Add opt-in "FOR NO KEY UPDATE SKIP LOCKED" job lock strategy and hybrid strategy for online migration [\#1731](https://github.com/bensheldon/good_job/pull/1731) ([bensheldon](https://github.com/bensheldon))
|
|
10
|
+
- Allow ordering by scheduled\_at instead of created\_at when dequeueing job [\#1645](https://github.com/bensheldon/good_job/pull/1645) ([lsylvester](https://github.com/lsylvester))
|
|
11
|
+
- Allow `GoodJob.preserve_job_records` to take a lambda that is callable after each job executes [\#1640](https://github.com/bensheldon/good_job/pull/1640) ([bensheldon](https://github.com/bensheldon))
|
|
12
|
+
|
|
13
|
+
**Merged pull requests:**
|
|
14
|
+
|
|
15
|
+
- Fix JRuby in development lockfile, with test [\#1734](https://github.com/bensheldon/good_job/pull/1734) ([bensheldon](https://github.com/bensheldon))
|
|
16
|
+
- Add herb to linter [\#1732](https://github.com/bensheldon/good_job/pull/1732) ([bensheldon](https://github.com/bensheldon))
|
|
17
|
+
- Update development dependencies; apply Rubocop to\_h lints [\#1728](https://github.com/bensheldon/good_job/pull/1728) ([bensheldon](https://github.com/bensheldon))
|
|
18
|
+
|
|
3
19
|
## [v4.14.2](https://github.com/bensheldon/good_job/tree/v4.14.2) (2026-04-06)
|
|
4
20
|
|
|
5
21
|
[Full Changelog](https://github.com/bensheldon/good_job/compare/v4.14.1...v4.14.2)
|
data/README.md
CHANGED
|
@@ -303,13 +303,13 @@ Available configuration options are:
|
|
|
303
303
|
- `cron_graceful_restart_period` (integer) when restarting cron, attempt to re-enqueue jobs that would have been enqueued by cron within this time period (e.g. `1.minute`). This should match the expected downtime during deploys.
|
|
304
304
|
- `enable_listen_notify` (boolean) whether to enqueue and read jobs with Postgres LISTEN/NOTIFY. Defaults to `true`. You can also set this with the environment variable `GOOD_JOB_ENABLE_LISTEN_NOTIFY`.
|
|
305
305
|
- `cron` (hash) cron configuration. Defaults to `{}`. You can also set this as a JSON string with the environment variable `GOOD_JOB_CRON`
|
|
306
|
-
- `cleanup_discarded_jobs` (boolean) whether to destroy discarded jobs when cleaning up preserved jobs using the `$ good_job cleanup_preserved_jobs` CLI command or calling `GoodJob.cleanup_preserved_jobs`. Defaults to `true`. Can also be set with
|
|
307
|
-
- `cleanup_preserved_jobs_before_seconds_ago` (integer) number of seconds to preserve jobs when using the `$ good_job cleanup_preserved_jobs` CLI command or calling `GoodJob.cleanup_preserved_jobs`. Defaults to `1209600` (14 days). Can also be set with the environment variable `GOOD_JOB_CLEANUP_PRESERVED_JOBS_BEFORE_SECONDS_AGO`.
|
|
306
|
+
- `cleanup_discarded_jobs` (boolean) whether to destroy discarded jobs when cleaning up preserved jobs using the `$ good_job cleanup_preserved_jobs` CLI command or calling `GoodJob.cleanup_preserved_jobs`. Defaults to `true`. Can also be set with the environment variable `GOOD_JOB_CLEANUP_DISCARDED_JOBS`.
|
|
307
|
+
- `cleanup_preserved_jobs_before_seconds_ago` (integer) number of seconds to preserve jobs when using the `$ good_job cleanup_preserved_jobs` CLI command or calling `GoodJob.cleanup_preserved_jobs`. Defaults to `1209600` (14 days). Can also be set with the environment variable `GOOD_JOB_CLEANUP_PRESERVED_JOBS_BEFORE_SECONDS_AGO`.
|
|
308
308
|
- `cleanup_interval_jobs` (integer) Number of jobs a Scheduler will execute before cleaning up preserved jobs. Defaults to `1000`. Disable with `false`. Can also be set with the environment variable `GOOD_JOB_CLEANUP_INTERVAL_JOBS` and disabled with `0`).
|
|
309
309
|
- `cleanup_interval_seconds` (integer) Number of seconds a Scheduler will wait before cleaning up preserved jobs. Defaults to `600` (10 minutes). Disable with `false`. Can also be set with the environment variable `GOOD_JOB_CLEANUP_INTERVAL_SECONDS` and disabled with `0`).
|
|
310
310
|
- `inline_execution_respects_schedule` (boolean) Opt-in to future behavior of inline execution respecting scheduled jobs. Defaults to `false`.
|
|
311
311
|
- `logger` ([Rails Logger](https://api.rubyonrails.org/classes/ActiveSupport/Logger.html)) lets you set a custom logger for GoodJob. It should be an instance of a Rails `Logger` (Default: `Rails.logger`).
|
|
312
|
-
- `preserve_job_records` (boolean) keeps job records in your database even after jobs are completed. (Default: `true`)
|
|
312
|
+
- `preserve_job_records` (boolean, symbol, or lambda) keeps job records in your database even after jobs are completed. If set to `true`, all job records are preserved. If set to `:on_unhandled_error`, only jobs that finished with an unhandled error are preserved. If set to a lambda, the lambda will be called with the error_event (e.g., `:discarded`, `:retry_stopped`, or `:unhandled`) and should return a boolean indicating whether to preserve the job. (Default: `true`)
|
|
313
313
|
- `advisory_lock_heartbeat` (boolean) whether to use an advisory lock for the purpose of determining whether an execeution process is active. (Default `true` in Development; `false` in other environments)
|
|
314
314
|
- `retry_on_unhandled_error` (boolean) causes jobs to be re-queued and retried if they raise an instance of `StandardError`. Be advised this may lead to jobs being repeated infinitely ([see below for more on retries](#retries)). Instances of `Exception`, like SIGINT, will *always* be retried, regardless of this attribute’s value. (Default: `false`)
|
|
315
315
|
- `on_thread_error` (proc, lambda, or callable) will be called when there is an Exception. It can be useful for logging errors to bug tracking services, like Sentry or Airbrake. Example:
|
|
@@ -375,7 +375,7 @@ GoodJob.active_record_parent_class = "ApplicationRecord"
|
|
|
375
375
|
The following options are also configurable via accessors, but you are encouraged to use the configuration attributes instead because these may be deprecated and removed in the future:
|
|
376
376
|
|
|
377
377
|
- **`GoodJob.logger`** ([Rails Logger](https://api.rubyonrails.org/classes/ActiveSupport/Logger.html)) lets you set a custom logger for GoodJob. It should be an instance of a Rails `Logger`.
|
|
378
|
-
- **`GoodJob.preserve_job_records`** (boolean) keeps job records in your database even after jobs are completed. (Default: `true`)
|
|
378
|
+
- **`GoodJob.preserve_job_records`** (boolean, symbol, or lambda) keeps job records in your database even after jobs are completed. If set to `true`, all job records are preserved. If set to `:on_unhandled_error`, only jobs that finished with an unhandled error are preserved. If set to a lambda, the lambda will be called with Active Job instance, and if it exists, the exception the error_event (e.g., `:discarded`, `:retry_stopped`, or `:unhandled`) and should return a boolean indicating whether to preserve the job (e.g. `-> (active_job, error, error_event) { !(active_job.is_a(Turbo::Streams::BroadcastStreamJob) || error_event == :retry_stopped`). (Default: `true`)
|
|
379
379
|
- **`GoodJob.retry_on_unhandled_error`** (boolean) causes jobs to be re-queued and retried if they raise an instance of `StandardError`. Be advised this may lead to jobs being repeated infinitely ([see below for more on retries](#retries)). Instances of `Exception`, like SIGINT, will *always* be retried, regardless of this attribute’s value. (Default: `false`)
|
|
380
380
|
- **`GoodJob.on_thread_error`** (proc, lambda, or callable) will be called when there is an Exception. It can be useful for logging errors to bug tracking services, like Sentry or Airbrake.
|
|
381
381
|
|
|
@@ -1390,7 +1390,11 @@ To instead delete job records immediately after they are finished:
|
|
|
1390
1390
|
|
|
1391
1391
|
```ruby
|
|
1392
1392
|
# config/initializers/good_job.rb
|
|
1393
|
-
config.good_job.preserve_job_records = false # defaults to true; can also be `false
|
|
1393
|
+
config.good_job.preserve_job_records = false # defaults to true; can also be `false`, `:on_unhandled_error`, or a lambda that takes error_event argument
|
|
1394
|
+
|
|
1395
|
+
# Example of using a lambda to preserve only discarded jobs
|
|
1396
|
+
config.good_job.preserve_job_records = ->(error_event) { error_event == :discarded }
|
|
1397
|
+
|
|
1394
1398
|
```
|
|
1395
1399
|
|
|
1396
1400
|
GoodJob will automatically delete preserved job records after 14 days. The retention period, as well as the frequency GoodJob checks for deletable records can be configured:
|
|
@@ -1433,6 +1437,53 @@ travel_to(15.minutes.from_now) { GoodJob.perform_inline }
|
|
|
1433
1437
|
|
|
1434
1438
|
_Note: Rails `travel`/`travel_to` time helpers do not have millisecond precision, so you must leave at least 1 second between the schedule and time traveling for the job to be executed. This [behavior may change in Rails 7.1](https://github.com/rails/rails/pull/44088)._
|
|
1435
1439
|
|
|
1440
|
+
### SKIP LOCKED experimental mode
|
|
1441
|
+
|
|
1442
|
+
By default, GoodJob claims jobs using PostgreSQL advisory locks. As an alternative, GoodJob can use `SELECT FOR UPDATE SKIP LOCKED` to claim jobs, which writes the lock state directly to the `good_jobs` table rather than relying on session-level advisory locks.
|
|
1443
|
+
|
|
1444
|
+
Two strategies are available:
|
|
1445
|
+
|
|
1446
|
+
- **`:skiplocked`** — Claims jobs using `SELECT FOR UPDATE SKIP LOCKED` only. No advisory locks are held. Compatible with PgBouncer in transaction mode.
|
|
1447
|
+
- **`:hybrid`** — Claims jobs using `SELECT FOR UPDATE SKIP LOCKED` and _also_ acquires a session-level advisory lock on the job. Intended for rolling deploys where some workers are still using the default `:advisory` strategy.
|
|
1448
|
+
|
|
1449
|
+
Configure the lock strategy in an initializer or via environment variable:
|
|
1450
|
+
|
|
1451
|
+
```ruby
|
|
1452
|
+
# config/initializers/good_job.rb
|
|
1453
|
+
GoodJob.configure do |config|
|
|
1454
|
+
config.lock_strategy = :skiplocked
|
|
1455
|
+
end
|
|
1456
|
+
```
|
|
1457
|
+
|
|
1458
|
+
```bash
|
|
1459
|
+
GOOD_JOB_LOCK_STRATEGY=skiplocked
|
|
1460
|
+
```
|
|
1461
|
+
|
|
1462
|
+
All three strategies (`:advisory`, `:skiplocked`, `:hybrid`) can coexist safely during a rolling deploy — each strategy excludes jobs that are already locked by another worker regardless of which strategy that worker uses.
|
|
1463
|
+
|
|
1464
|
+
#### PgBouncer configuration
|
|
1465
|
+
|
|
1466
|
+
GoodJob's `:skiplocked` mode makes it compatible with PgBouncer in _transaction_ mode. In addition to setting the lock strategy, you must also disable the `LISTEN/NOTIFY` notifier (which requires a persistent connection) and rely on polling instead:
|
|
1467
|
+
|
|
1468
|
+
```ruby
|
|
1469
|
+
# config/initializers/good_job.rb
|
|
1470
|
+
GoodJob.configure do |config|
|
|
1471
|
+
config.lock_strategy = :skiplocked
|
|
1472
|
+
config.enable_listen_notify = false
|
|
1473
|
+
config.advisory_lock_heartbeat = false
|
|
1474
|
+
config.poll_interval = 5 # seconds; tune based on your latency tolerance
|
|
1475
|
+
end
|
|
1476
|
+
```
|
|
1477
|
+
|
|
1478
|
+
```bash
|
|
1479
|
+
GOOD_JOB_LOCK_STRATEGY=skiplocked
|
|
1480
|
+
GOOD_JOB_ENABLE_LISTEN_NOTIFY=false
|
|
1481
|
+
GOOD_JOB_ADVISORY_LOCK_HEARTBEAT=false
|
|
1482
|
+
GOOD_JOB_POLL_INTERVAL=5
|
|
1483
|
+
```
|
|
1484
|
+
|
|
1485
|
+
With these four settings, GoodJob will not hold any session-level state between queries and is safe to use behind PgBouncer in transaction mode.
|
|
1486
|
+
|
|
1436
1487
|
### PgBouncer compatibility
|
|
1437
1488
|
|
|
1438
1489
|
GoodJob is not compatible with PgBouncer in _transaction_ mode, but is compatible with PgBouncer's _connection_ mode. GoodJob uses connection-based advisory locks and LISTEN/NOTIFY, both of which require full database connections.
|
|
@@ -7,7 +7,7 @@ module GoodJob
|
|
|
7
7
|
|
|
8
8
|
@discarded_jobs_grouped_by_exception =
|
|
9
9
|
GoodJob::Job.discarded
|
|
10
|
-
.select(
|
|
10
|
+
.select(<<~SQL.squish)
|
|
11
11
|
SPLIT_PART(error, ': ', 1) AS exception_class,
|
|
12
12
|
count(id) AS failed,
|
|
13
13
|
COUNT(id) FILTER (WHERE "finished_at" > NOW() - INTERVAL '1 HOUR') AS last_1_hour,
|
|
@@ -21,7 +21,7 @@ module GoodJob
|
|
|
21
21
|
|
|
22
22
|
@discarded_jobs_grouped_by_class =
|
|
23
23
|
GoodJob::Job.discarded
|
|
24
|
-
.select(
|
|
24
|
+
.select(<<~SQL.squish)
|
|
25
25
|
job_class,
|
|
26
26
|
count(id) AS failed,
|
|
27
27
|
COUNT(*) FILTER (WHERE "finished_at" > NOW() - INTERVAL '1 HOUR') AS last_1_hour,
|
|
@@ -5,14 +5,21 @@ module GoodJob
|
|
|
5
5
|
module ErrorEvents
|
|
6
6
|
extend ActiveSupport::Concern
|
|
7
7
|
|
|
8
|
+
INTERRUPTED = :interrupted
|
|
9
|
+
UNHANDLED = :unhandled
|
|
10
|
+
HANDLED = :handled
|
|
11
|
+
RETRIED = :retried
|
|
12
|
+
RETRY_STOPPED = :retry_stopped
|
|
13
|
+
DISCARDED = :discarded
|
|
14
|
+
|
|
8
15
|
included do
|
|
9
16
|
error_event_enum = {
|
|
10
|
-
|
|
11
|
-
|
|
12
|
-
|
|
13
|
-
|
|
14
|
-
|
|
15
|
-
|
|
17
|
+
INTERRUPTED => 0,
|
|
18
|
+
UNHANDLED => 1,
|
|
19
|
+
HANDLED => 2,
|
|
20
|
+
RETRIED => 3,
|
|
21
|
+
RETRY_STOPPED => 4,
|
|
22
|
+
DISCARDED => 5,
|
|
16
23
|
}
|
|
17
24
|
if Gem::Version.new(Rails.version) >= Gem::Version.new('7.1.0.a')
|
|
18
25
|
enum :error_event, error_event_enum, validate: { allow_nil: true }, scopes: false
|
|
@@ -0,0 +1,75 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
module GoodJob
|
|
4
|
+
# Adds row-level locking capabilities (SKIP LOCKED) to ActiveRecord models.
|
|
5
|
+
# These methods provide strategy-agnostic job claiming via CTE UPDATE statements.
|
|
6
|
+
module Lockable
|
|
7
|
+
extend ActiveSupport::Concern
|
|
8
|
+
|
|
9
|
+
class_methods do
|
|
10
|
+
# Claims a job by acquiring a session-level advisory lock.
|
|
11
|
+
# Returns the claimed record or nil if no eligible record was found.
|
|
12
|
+
# The caller is responsible for releasing the advisory lock after use.
|
|
13
|
+
# @param select_limit [Integer, nil] Number of candidates to attempt locking
|
|
14
|
+
# @return [ActiveRecord::Base, nil]
|
|
15
|
+
def with_advisory_lock_claim(select_limit: nil)
|
|
16
|
+
advisory_lock(select_limit: select_limit).first
|
|
17
|
+
end
|
|
18
|
+
|
|
19
|
+
# Atomically claims a job using SELECT FOR NO KEY UPDATE SKIP LOCKED in a CTE UPDATE.
|
|
20
|
+
# Returns the claimed record or nil if no eligible record was found.
|
|
21
|
+
# @param locked_by_id [String] The process UUID claiming the job
|
|
22
|
+
# @param locked_at [Time] When the job was claimed
|
|
23
|
+
# @param lock_type [String, Symbol] Lock type identifier
|
|
24
|
+
# @return [ActiveRecord::Base, nil]
|
|
25
|
+
def with_skip_locked_claim(locked_by_id:, locked_at:, lock_type:)
|
|
26
|
+
candidate_sql = select(:id).lock("FOR NO KEY UPDATE SKIP LOCKED").to_sql
|
|
27
|
+
quoted_table = adapter_class.quote_table_name(table_name)
|
|
28
|
+
materialized = supports_cte_materialization_specifiers? ? "MATERIALIZED " : ""
|
|
29
|
+
|
|
30
|
+
sql = <<~SQL.squish
|
|
31
|
+
WITH candidate AS #{materialized}(#{candidate_sql})
|
|
32
|
+
UPDATE #{quoted_table}
|
|
33
|
+
SET locked_by_id = ?,
|
|
34
|
+
locked_at = ?,
|
|
35
|
+
lock_type = ?
|
|
36
|
+
FROM candidate
|
|
37
|
+
WHERE #{quoted_table}.id = candidate.id
|
|
38
|
+
RETURNING #{quoted_table}.*
|
|
39
|
+
SQL
|
|
40
|
+
|
|
41
|
+
unscoped.find_by_sql([sql, locked_by_id, locked_at, lock_types[lock_type.to_s]]).first
|
|
42
|
+
end
|
|
43
|
+
|
|
44
|
+
# Atomically claims a job using SELECT FOR NO KEY UPDATE SKIP LOCKED with an
|
|
45
|
+
# additional session-level advisory lock acquired within the same statement.
|
|
46
|
+
# Returns the claimed record or nil if no eligible record was found.
|
|
47
|
+
# @param locked_by_id [String] The process UUID claiming the job
|
|
48
|
+
# @param locked_at [Time] When the job was claimed
|
|
49
|
+
# @param lock_type [String, Symbol] Lock type identifier
|
|
50
|
+
# @return [ActiveRecord::Base, nil]
|
|
51
|
+
def with_hybrid_lock_claim(locked_by_id:, locked_at:, lock_type:)
|
|
52
|
+
candidate_sql = select(:id).lock("FOR NO KEY UPDATE SKIP LOCKED").to_sql
|
|
53
|
+
quoted_table = adapter_class.quote_table_name(table_name)
|
|
54
|
+
advisory_lock_expr = "('x' || substr(md5(#{_quoted_table_name_string} || '-' || id::text), 1, 16))::bit(64)::bigint"
|
|
55
|
+
materialized = supports_cte_materialization_specifiers? ? "MATERIALIZED " : ""
|
|
56
|
+
|
|
57
|
+
sql = <<~SQL.squish
|
|
58
|
+
WITH candidate AS #{materialized}(#{candidate_sql})
|
|
59
|
+
UPDATE #{quoted_table}
|
|
60
|
+
SET locked_by_id = ?,
|
|
61
|
+
locked_at = ?,
|
|
62
|
+
lock_type = ?
|
|
63
|
+
FROM (
|
|
64
|
+
SELECT id FROM candidate
|
|
65
|
+
WHERE pg_try_advisory_lock(#{advisory_lock_expr})
|
|
66
|
+
) AS locked
|
|
67
|
+
WHERE #{quoted_table}.id = locked.id
|
|
68
|
+
RETURNING #{quoted_table}.*
|
|
69
|
+
SQL
|
|
70
|
+
|
|
71
|
+
unscoped.find_by_sql([sql, locked_by_id, locked_at, lock_types[lock_type.to_s]]).first
|
|
72
|
+
end
|
|
73
|
+
end
|
|
74
|
+
end
|
|
75
|
+
end
|
|
@@ -94,60 +94,90 @@ module GoodJob
|
|
|
94
94
|
# Phase 2: Build and partition jobs by concurrency limits
|
|
95
95
|
build_result = _build_and_partition_jobs(batch_job_pairs, current_time)
|
|
96
96
|
|
|
97
|
-
# Phase 3: Insert
|
|
97
|
+
# Phase 3–6: Insert, claim, and execute inline jobs
|
|
98
|
+
lock_strategy = Job.effective_lock_strategy
|
|
99
|
+
tracker_registered = false
|
|
100
|
+
lock_id = nil
|
|
98
101
|
persisted_jobs = []
|
|
99
102
|
inline_jobs = []
|
|
100
103
|
|
|
101
|
-
if
|
|
102
|
-
|
|
103
|
-
|
|
104
|
+
if execute_inline
|
|
105
|
+
GoodJob.capsule.tracker.register
|
|
106
|
+
tracker_registered = true
|
|
107
|
+
lock_id = GoodJob.capsule.tracker.id_for_lock
|
|
108
|
+
end
|
|
104
109
|
|
|
105
|
-
|
|
106
|
-
|
|
107
|
-
|
|
110
|
+
begin
|
|
111
|
+
if build_result[:bulkable].any?
|
|
112
|
+
Job.transaction(requires_new: true, joinable: false) do
|
|
113
|
+
persisted_jobs = _insert_jobs(build_result[:bulkable], build_result[:active_jobs_by_job_id])
|
|
114
|
+
|
|
115
|
+
if execute_inline
|
|
116
|
+
inline_jobs = persisted_jobs.select { |job| job.scheduled_at.nil? || job.scheduled_at <= current_time }
|
|
117
|
+
if lock_strategy != :advisory && lock_id && inline_jobs.any?
|
|
118
|
+
Job.where(id: inline_jobs.map(&:id)).update_all( # rubocop:disable Rails/SkipsModelValidations
|
|
119
|
+
locked_by_id: lock_id, locked_at: current_time, lock_type: Job.lock_types[lock_strategy.to_s]
|
|
120
|
+
)
|
|
121
|
+
inline_jobs.each { |j| j.assign_attributes(locked_by_id: lock_id, locked_at: current_time, lock_type: lock_strategy) }
|
|
122
|
+
end
|
|
123
|
+
case lock_strategy
|
|
124
|
+
when :advisory, :hybrid
|
|
125
|
+
inline_jobs.each(&:advisory_lock!)
|
|
126
|
+
end
|
|
127
|
+
end
|
|
108
128
|
end
|
|
109
129
|
end
|
|
110
|
-
end
|
|
111
130
|
|
|
112
|
-
|
|
113
|
-
|
|
114
|
-
|
|
115
|
-
|
|
116
|
-
|
|
117
|
-
|
|
118
|
-
|
|
119
|
-
|
|
120
|
-
|
|
121
|
-
|
|
122
|
-
|
|
123
|
-
|
|
131
|
+
# Phase 4: Handle empty batches — they need _continue_discard_or_finish
|
|
132
|
+
# to trigger on_success/on_finish callbacks (batch_record.rb:77).
|
|
133
|
+
batches_with_jobs = Set.new
|
|
134
|
+
build_result[:bulkable].each { |entry| batches_with_jobs.add(entry[:batch]) }
|
|
135
|
+
build_result[:unbulkable].each { |entry| batches_with_jobs.add(entry[:batch]) }
|
|
136
|
+
|
|
137
|
+
empty_batches = batch_job_pairs.map(&:first).reject { |batch| batches_with_jobs.include?(batch) }
|
|
138
|
+
if empty_batches.any?
|
|
139
|
+
buffer = GoodJob::Adapter::InlineBuffer.capture do
|
|
140
|
+
empty_batches.each do |batch|
|
|
141
|
+
batch._record.reload
|
|
142
|
+
batch._record._continue_discard_or_finish(lock: true)
|
|
143
|
+
end
|
|
124
144
|
end
|
|
145
|
+
buffer.call
|
|
125
146
|
end
|
|
126
|
-
buffer.call
|
|
127
|
-
end
|
|
128
147
|
|
|
129
|
-
|
|
130
|
-
|
|
131
|
-
|
|
132
|
-
|
|
148
|
+
# Phase 5: Enqueue concurrency-limited jobs individually
|
|
149
|
+
build_result[:unbulkable].each do |entry|
|
|
150
|
+
within_thread(batch_id: entry[:batch].id) do
|
|
151
|
+
entry[:active_job].enqueue
|
|
152
|
+
end
|
|
153
|
+
rescue GoodJob::ActiveJobExtensions::Concurrency::ConcurrencyExceededError
|
|
154
|
+
# ignore — matches Bulk::Buffer behavior (bulk.rb:107-109)
|
|
133
155
|
end
|
|
134
|
-
rescue GoodJob::ActiveJobExtensions::Concurrency::ConcurrencyExceededError
|
|
135
|
-
# ignore — matches Bulk::Buffer behavior (bulk.rb:107-109)
|
|
136
|
-
end
|
|
137
156
|
|
|
138
|
-
|
|
139
|
-
|
|
140
|
-
|
|
141
|
-
GoodJob.
|
|
157
|
+
# Phase 6: Execute inline jobs
|
|
158
|
+
if inline_jobs.any?
|
|
159
|
+
deferred = GoodJob::Adapter::InlineBuffer.defer?
|
|
160
|
+
GoodJob::Adapter::InlineBuffer.perform_now_or_defer do
|
|
142
161
|
until inline_jobs.empty?
|
|
143
162
|
inline_job = inline_jobs.shift
|
|
144
163
|
active_job = build_result[:active_jobs_by_job_id][inline_job.active_job_id]
|
|
145
|
-
adapter.send(:perform_inline, inline_job, notify: adapter.send(:send_notify?, active_job))
|
|
164
|
+
adapter.send(:perform_inline, inline_job, notify: deferred ? adapter.send(:send_notify?, active_job) : false, already_claimed: lock_strategy != :advisory, advisory_unlock: lock_strategy != :skiplocked)
|
|
146
165
|
end
|
|
147
166
|
ensure
|
|
148
167
|
inline_jobs.each(&:advisory_unlock)
|
|
168
|
+
GoodJob.capsule.tracker.unregister if tracker_registered
|
|
169
|
+
tracker_registered = false
|
|
149
170
|
end
|
|
171
|
+
elsif tracker_registered
|
|
172
|
+
GoodJob.capsule.tracker.unregister
|
|
173
|
+
tracker_registered = false
|
|
174
|
+
end
|
|
175
|
+
rescue StandardError
|
|
176
|
+
if tracker_registered
|
|
177
|
+
GoodJob.capsule.tracker.unregister
|
|
178
|
+
tracker_registered = false
|
|
150
179
|
end
|
|
180
|
+
raise
|
|
151
181
|
end
|
|
152
182
|
|
|
153
183
|
# Phase 7: Send NOTIFY for non-inline jobs
|
|
@@ -360,10 +390,11 @@ module GoodJob
|
|
|
360
390
|
|
|
361
391
|
# @!visibility private
|
|
362
392
|
def self._insert_jobs(bulkable_entries, active_jobs_by_job_id)
|
|
363
|
-
|
|
393
|
+
column_names = Job.column_names
|
|
394
|
+
job_attributes = bulkable_entries.map { |entry| entry[:good_job].attributes.slice(*column_names) }
|
|
364
395
|
results = Job.insert_all(job_attributes, returning: %w[id active_job_id]) # rubocop:disable Rails/SkipsModelValidations
|
|
365
396
|
|
|
366
|
-
job_id_map = results.
|
|
397
|
+
job_id_map = results.to_h { |row| [row['active_job_id'], row['id']] }
|
|
367
398
|
|
|
368
399
|
# Set provider_job_id on ActiveJob instances (mirrors adapter.rb:74-76)
|
|
369
400
|
active_jobs_by_job_id.each_value do |active_job|
|
|
@@ -3,6 +3,8 @@
|
|
|
3
3
|
module GoodJob
|
|
4
4
|
# Stores the results of job execution
|
|
5
5
|
class ExecutionResult
|
|
6
|
+
# @return [ActiveJob::Base, nil]
|
|
7
|
+
attr_reader :active_job
|
|
6
8
|
# @return [Object, nil]
|
|
7
9
|
attr_reader :value
|
|
8
10
|
# @return [Exception, nil]
|
|
@@ -22,8 +24,9 @@ module GoodJob
|
|
|
22
24
|
# @param error_event [String, nil]
|
|
23
25
|
# @param unexecutable [Boolean, nil]
|
|
24
26
|
# @param retried_job [GoodJob::Job, nil]
|
|
25
|
-
def initialize(value:, handled_error: nil, unhandled_error: nil, error_event: nil, unexecutable: nil, retried_job: nil)
|
|
27
|
+
def initialize(value:, active_job: nil, handled_error: nil, unhandled_error: nil, error_event: nil, unexecutable: nil, retried_job: nil)
|
|
26
28
|
@value = value
|
|
29
|
+
@active_job = active_job
|
|
27
30
|
@handled_error = handled_error
|
|
28
31
|
@unhandled_error = unhandled_error
|
|
29
32
|
@error_event = error_event
|